Alessio Cecchi
2021-Oct-25 19:33 UTC
Doveadm fetch slow and 100%CPU with a specific message-id
Il 25/10/21 19:05, Aki Tuomi ha scritto:>> On 25/10/2021 19:40 Alessio Cecchi <alessio at skye.it> wrote: >> >> >> Hi, >> I'm using doveadm fetch in order to find the mailbox where a messagge is stored: >> doveadm fetch -u alessio at domain.com "mailbox" HEADER Message-ID '1234 at domain.com' >> If the messagge-id is ... long more than? ... I don't know, the lookup is very very slow, here an example: >> with message-id 9c102380c557e7e146a33cb4b49ab802 at cbweb.cecchi.net respons time: 3 secs >> with message-id kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E at www.myxmail.com response time: 80 secs and java/solr use the 100% CPU >> Both messages are in the same folder (Trash) >> >> If I add -D to doveadm it stuck some seconds every time connect to Solr: >> >> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: queue http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E at www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio at domain.com] >> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: conn 127.0.0.1:8983 [1]: Claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E at www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio at domain.com] >> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: request [Req1: GET http://127.0.0.1/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E at www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio at domain...: Sent header >> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: peer 127.0.0.1:8983: No more requests to service for this peer (1 connections exist, 0 pending) >> [...] >> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: queue http://127.0.0.1:8983 >> My dovecot version is 2.3.16 and Solr 7.7. >> >> Why? >> Thanks >> >> -- >> Alessio Cecchi >> Postmaster @ http://www.qboxmail.it >> https://www.linkedin.com/in/alessice > > I wasn't able to reproduce this issue locally. Maybe your solr config has issues? > > AkiYes, could be my Solr setup. I'll investigate. Is there a way to disable fts/solr when using doveadm? Like -o "plugin/fts=" Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
Alessio Cecchi
2021-Oct-25 19:43 UTC
Doveadm fetch slow and 100%CPU with a specific message-id
Il 25/10/21 21:33, Alessio Cecchi ha scritto:> Il 25/10/21 19:05, Aki Tuomi ha scritto: >>> On 25/10/2021 19:40 Alessio Cecchi <alessio at skye.it> wrote: >>> >>> >>> Hi, >>> I'm using doveadm fetch in order to find the mailbox where a >>> messagge is stored: >>> doveadm fetch -u alessio at domain.com "mailbox" HEADER Message-ID >>> '1234 at domain.com' >>> If the messagge-id is ... long more than? ... I don't know, the >>> lookup is very very slow, here an example: >>> with message-id 9c102380c557e7e146a33cb4b49ab802 at cbweb.cecchi.net >>> respons time: 3 secs >>> with message-id >>> kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E at www.myxmail.com response >>> time: 80 secs and java/solr use the 100% CPU >>> Both messages are in the same folder (Trash) >>> >>> If I add -D to doveadm it stuck some seconds every time connect to >>> Solr: >>> >>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: >>> queue http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 >>> claimed request [Req1: GET >>> http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E at www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio at domain.com] >>> ? Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: >>> conn 127.0.0.1:8983 [1]: Claimed request [Req1: GET >>> http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E at www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio at domain.com] >>> ? Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: >>> request [Req1: GET >>> http://127.0.0.1/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E at www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio at domain...: >>> Sent header >>> ? Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: >>> peer 127.0.0.1:8983: No more requests to service for this peer (1 >>> connections exist, 0 pending) >>> [...] >>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: >>> queue http://127.0.0.1:8983 >>> My dovecot version is 2.3.16 and Solr 7.7. >>> >>> Why? >>> Thanks >>> >>> -- >>> Alessio Cecchi >>> Postmaster @ http://www.qboxmail.it >>> https://www.linkedin.com/in/alessice >> >> I wasn't able to reproduce this issue locally. Maybe your solr config >> has issues? >> >> Aki > > Yes, could be my Solr setup. I'll investigate. > > Is there a way to disable fts/solr when using doveadm? Like -o > "plugin/fts="Yes, with: doveadm -o "plugin/fts=" fetch -u alessio at ... the time needed to lookup the message is now around 3 seconds for all query. But, is possible to use the -o "plugin/fts=" in doveadm http API? Because I run doveadm via http API with curl. Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice