dovecot - Oct 2021 - Doveadm fetch slow and 100%CPU with a specific message-id

On Oct 25, 2021, at 10:05 AM, Aki Tuomi <aki.tuomi at open-xchange.com>
wrote:
> 
>> On 25/10/2021 19:40 Alessio Cecchi <alessio at skye.it> wrote:
>> 
>> 
>> Hi,
>> I'm using doveadm fetch in order to find the mailbox where a
messagge is stored:
>> doveadm fetch -u alessio at domain.com "mailbox" HEADER
Message-ID '1234 at domain.com'
>> If the messagge-id is ... long more than? ... I don't know, the
lookup is very very slow, here an example:
>> with message-id 9c102380c557e7e146a33cb4b49ab802 at cbweb.cecchi.net
respons time: 3 secs
>> with message-id kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E at
www.myxmail.com response time: 80 secs and java/solr use the 100% CPU
>> Both messages are in the same folder (Trash)
>> 
>> If I add -D to doveadm it stuck some seconds every time connect to
Solr:
>> 
>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client:
queue http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed request
[Req1: GET
http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E
at
www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio
at domain.com]
>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client:
conn 127.0.0.1:8983 [1]: Claimed request [Req1: GET
http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E
at
www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio
at domain.com]
>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client:
request [Req1: GET
http://127.0.0.1/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E
at
www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio
at domain...: Sent header
>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client:
peer 127.0.0.1:8983: No more requests to service for this peer (1 connections
exist, 0 pending)
>> [...]
>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client:
queue http://127.0.0.1:8983
>> My dovecot version is 2.3.16 and Solr 7.7.
>> 
>> Why?
>> Thanks
>> 
>> -- 
>> Alessio Cecchi
>> Postmaster @ http://www.qboxmail.it
>> https://www.linkedin.com/in/alessice
> 
> 
> I wasn't able to reproduce this issue locally. Maybe your solr config
has issues?
I also tried and failed to reproduce this problem (in a folder with 1500
messages).

Note that message-ids are not guaranteed to be unique.  During my test I found
groups of as many as 20 different messages with the same message ID.  (Turns out
this makes quite a reliable spam signal!)

rg

On Mon, 25 Oct 2021, Ron Garret wrote:
> Note that message-ids are not guaranteed to be unique.  During my test
> I found groups of as many as 20 different messages with the same
> message ID.  (Turns out this makes quite a reliable spam signal!)
It's by far not a rare situation: duplicate message-ids happen whenever
the sender names more than one local recipient during SMTP.  It's a wholly
unreliable way to indicates spaminess.  However, if a high proportion
of those recipients do not exist, ...

Joseph Tam <jtam.home at gmail.com>