dovecot - Sep 2020 - Sieve: deleteheader not working with duplicate filter for implicit keep

Hello,

I have a rule to always delete a header. If the message gets fileinto'd,
the header is gone, but if it's delivered into the INBOX through
implicit keep, the header does not get deleted.

Interestingly, if I remove the "if duplicate" filter, the header does
get deleted as expected.

I'm using the latest Dovecot 2.3.11.3 / Pigeonhole 0.5.11.

Any idea why this is happening?

Thanks,
Alec

=== Sieve filter

require ["fileinto", "editheader", "duplicate",
"mailbox"];

deleteheader "X-Bad-Header";

if duplicate {
        fileinto :create "Trash";
        stop;
}

if header :matches "subject" "*filter*" {
    fileinto :create "Filtered";
}

=== Example (note ${RANDOM} to bypass duplicate filter)

$ /usr/libexec/dovecot/deliver << -end-
To: me at example.com
From: someone at example.com
Subject: Test Message
Message-Id: <${RANDOM}@example.net>
Date: Thu, 20 Aug 2020 00:21:24 +0000 (UTC)
X-Bad-Header: DELETE ME

Hello world
-end-

On 29/08/2020 21:04, Alec Moskvin wrote:
> Hello,
>
> I have a rule to always delete a header. If the message gets
fileinto'd,
> the header is gone, but if it's delivered into the INBOX through
> implicit keep, the header does not get deleted.
>
> Interestingly, if I remove the "if duplicate" filter, the header
does
> get deleted as expected.
>
> I'm using the latest Dovecot 2.3.11.3 / Pigeonhole 0.5.11.
>
> Any idea why this is happening?
Can you enable Sieve tracing? That should show you in detail what is 
happening during delivery and why.

Regards,

Stephan.
> Thanks,
> Alec
>
> === Sieve filter
>
> require ["fileinto", "editheader",
"duplicate", "mailbox"];
>
> deleteheader "X-Bad-Header";
>
> if duplicate {
>          fileinto :create "Trash";
>          stop;
> }
>
> if header :matches "subject" "*filter*" {
>      fileinto :create "Filtered";
> }
>
> === Example (note ${RANDOM} to bypass duplicate filter)
>
> $ /usr/libexec/dovecot/deliver << -end-
> To: me at example.com
> From: someone at example.com
> Subject: Test Message
> Message-Id: <${RANDOM}@example.net>
> Date: Thu, 20 Aug 2020 00:21:24 +0000 (UTC)
> X-Bad-Header: DELETE ME
>
> Hello world
> -end-

Hi Stephan,

On Wednesday 02 September 2020 19:59:57, Stephan Bosch
wrote:
>
>
> On 29/08/2020 21:04, Alec Moskvin wrote:
> > Hello,
> >
> > I have a rule to always delete a header. If the message gets
fileinto'd,
> > the header is gone, but if it's delivered into the INBOX through
> > implicit keep, the header does not get deleted.
> >
> > Interestingly, if I remove the "if duplicate" filter, the
header does
> > get deleted as expected.
> >
> > I'm using the latest Dovecot 2.3.11.3 / Pigeonhole 0.5.11.
> >
> > Any idea why this is happening?
>
> Can you enable Sieve tracing? That should show you in detail what is
> happening during delivery and why.
It's saying "deleted 1 occurrences of header", but when it gets
stored,
the header is still there.


+ nl -ba mini.sieve
     1	require ["fileinto", "editheader",
"duplicate", "mailbox"];
     2
     3	deleteheader "X-Bad-Header";
     4
     5	if duplicate {
     6		fileinto :create "Trash";
     7		stop;
     8	}

+ tee test.msg
To: me at example.com
From: someone at example.com
Subject: Test Message
Message-Id: <8500 at example.net>
Date: Thu, 20 Aug 2020 00:21:24 +0000 (UTC)
X-Bad-Header: DELETE ME

Hello world

+ sieve-test -CDe -t- -Tlevel=matching -Tdebug -l maildir:maildir mini.sieve
test.msg
sieve-test(alec): Debug: sieve: Pigeonhole version 0.5.11 (d71e0372)
initializing
sieve-test(alec): Debug: sieve: include: sieve_global is not set; it is
currently not possible to include `:global' scripts.
sieve-test(alec): Debug: sieve: file storage: Using Sieve script path:
/home/alec/sievetest/header/mini.sieve
sieve-test(alec): Debug: sieve: file storage: script: Opened script `mini'
from `/home/alec/sievetest/header/mini.sieve'
sieve-test(alec): Debug: sieve: file storage: script: Opened script for reading
sieve-test(alec): Debug: sieve: Script `mini' from
/home/alec/sievetest/header/mini.sieve successfully compiled

      ## Started executing script 'mini'
sieve-test(alec): Debug: sieve: Started running script
`/home/alec/sievetest/header/mini.sieve'
   3: deleteheader command
   3:   deleting header `X-Bad-Header'
   3:   deleted 1 occurrences of header `X-Bad-Header'
   5: duplicate test
sieve-test(alec): Info: checked duplicate for user alec.

   5:   message is not a duplicate
   5: jump if result is false
   5:   jumping to line 7
sieve-test(alec): Debug: sieve: Finished running script
`/home/alec/sievetest/header/mini.sieve'
      ## Finished executing script 'mini'

sieve-test(alec): Info: sieve: msgid=<8500 at example.net>: stored mail
into mailbox 'INBOX'
info: msgid=<8500 at example.net>: stored mail into mailbox
'INBOX'.
sieve-test(alec): Info: marked duplicate for user alec.

sieve-test(alec): Info: final result: success
sieve-test(alec): Debug: sieve: file storage: script: Closed script

+ cat maildir/new/1599167523.M233933P55196.haswell,S=179,W=187
To: me at example.com
From: someone at example.com
Subject: Test Message
Message-Id: <8500 at example.net>
Date: Thu, 20 Aug 2020 00:21:24 +0000 (UTC)
X-Bad-Header: DELETE ME

Hello world