dovecot - May 2015 - Problem with sieve not triggering randomly?

I'm running CentOS 7.1 with Dovecot 2.2.10 (CentOS packaged version).  I
have Postfix feeding messages to Dovecot for local delivery via LMTP,
and I have Amavisd for spam filtering.  For messages above a certain
threshold, Amavisd adds the "X-Spam-Flag: YES" header, and I file that
into the Spam folder using a default (system-wide) sieve script (there
aren't any per-user sieve scripts set up):

require "fileinto";
if header :contains "X-Spam-Flag" "YES" {
	fileinto "Spam";
}

I can see it work on my test messages and from lmtp log entries.

However, it doesn't seem to always work.  I had a user complain about a
spam increase, and I looked in their mailbox.  I see a message with the
"X-Spam-Flag: YES" header, but it is in the INBOX (and the
corresponding
lmtp log entry shows that's where it went).

Why would the sieve script work on some messages and not others?  I
don't see any errors in the log.  I see a bunch of messages from the
same source to different users, and they were all filed into INBOX, not
Spam.

Looking at the message in question, it is a pretty bogus message.  There
are multiple From: and Subject: headers.  Would that cause sieve to
"punt" on the message (but not log anything about it)?

-- 
Chris Adams <cma at cmadams.net>

On 5/15/2015 4:58 PM, Chris Adams wrote:
> I'm running CentOS 7.1 with Dovecot 2.2.10 (CentOS packaged version). 
I
> have Postfix feeding messages to Dovecot for local delivery via LMTP,
> and I have Amavisd for spam filtering.  For messages above a certain
> threshold, Amavisd adds the "X-Spam-Flag: YES" header, and I file
that
> into the Spam folder using a default (system-wide) sieve script (there
> aren't any per-user sieve scripts set up):
>
> require "fileinto";
> if header :contains "X-Spam-Flag" "YES" {
> 	fileinto "Spam";
> }
>
> I can see it work on my test messages and from lmtp log entries.
>
> However, it doesn't seem to always work.  I had a user complain about a
> spam increase, and I looked in their mailbox.  I see a message with the
> "X-Spam-Flag: YES" header, but it is in the INBOX (and the
corresponding
> lmtp log entry shows that's where it went).
>
> Why would the sieve script work on some messages and not others?  I
> don't see any errors in the log.  I see a bunch of messages from the
> same source to different users, and they were all filed into INBOX, not
> Spam.
>
> Looking at the message in question, it is a pretty bogus message.  There
> are multiple From: and Subject: headers.  Would that cause sieve to
> "punt" on the message (but not log anything about it)?
You can check the handling of a particular message yourself using the
sieve-test tool (there is a man page for it). By specifying the `-t -
-Tlevel=matching` options, you'll get detailed information on why a
particular decision is made.

Without insight in your configuration (output from `dovecot -n`) and an
example of a message that is mishandled, I cannot tell you more.

Regards,

Stephan.

Once upon a time, Stephan Bosch <stephan at rename-it.nl>
said:
> You can check the handling of a particular message yourself using the
> sieve-test tool (there is a man page for it). By specifying the `-t -
> -Tlevel=matching` options, you'll get detailed information on why a
> particular decision is made.
Hmm, that's weird.  sieve-test says it would store the message into the
Spam folder (as expected), but it was definitely delivered to INBOX.
> Without insight in your configuration (output from `dovecot -n`) and an
> example of a message that is mishandled, I cannot tell you more.
Here you go.  I changed the local part of the user's email address in
the spam message, and the doveadm password in the doveconf output, but
that's all.

http://www.cmadams.net/misc/dove/doveconf-n.txt
http://www.cmadams.net/misc/dove/spam-message.txt

I have three Dovecot VMs behind a pair of Dovecot director VMs; this is
from the actual host that handled the delivery (all three backend hosts
are showing the same issue).  Users are in MySQL and mail is on NFS.
There are some things configured, like managesieve, that aren't
currently used.

This is a new setup, migrated from Courier; I made the switch a couple
of weeks ago (not my first time using Dovecot, just finally got a chance
to migrate this setup).

Thanks.
-- 
Chris Adams <cma at cmadams.net>