Hi! Does the duplicate sieve plugin do any locking to avoid duplicate parallel delivery of the same message? I sometimes experience duplicate mail delivery of messages with the same message-id, despite the use of a sieve duplicate filter. According to the log files, those messages are delivered in the same second by two parallel dovecot-lda processes. (Duplicate filtering works fine in all other cases.) RFC7352 states that the ID of a message may only be committed to the duplicate tracking list at the _end_ of a successful script execution, which may lead to race conditions. Maybe I am running into this? Is there an easy way to serialize mail delivery using some locking inside sieve? Or do I have to serialize per-user dovecot-lda delivery? Any experiences with that? I am using dovecot-2.2.25 and pidgeonhole-0.4.15. Mail is delivered using postfix-2.10 and dovecot-lda as mailbox_command. Mailbox format is maildir with LAYOUT=fs. Cheers, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria
Op 17-10-2016 om 18:05 schreef Alexander 'Leo' Bergolth:> Hi! > > Does the duplicate sieve plugin do any locking to avoid duplicate > parallel delivery of the same message? > > I sometimes experience duplicate mail delivery of messages with the same > message-id, despite the use of a sieve duplicate filter. According to > the log files, those messages are delivered in the same second by two > parallel dovecot-lda processes. (Duplicate filtering works fine in all > other cases.) > > RFC7352 states that the ID of a message may only be committed to the > duplicate tracking list at the _end_ of a successful script execution, > which may lead to race conditions. > Maybe I am running into this? > > Is there an easy way to serialize mail delivery using some locking > inside sieve?We've seen this before I think. It would require some changes to the duplicate tracking system. I'd expect the vacation command to be affected as well.> Or do I have to serialize per-user dovecot-lda delivery? Any experiences > with that?Very little. I know there is a new lmtp_user_concurrency_limit setting, but there is not much documentation apart from the commit message: https://github.com/dovecot/core/commit/42abccd9b2a5a4190bd3c14ec2dcc10d51c0f491 There are possibilities from within the MTA as well I expect. Regards, Stephan.
On 10/19/2016 12:51 PM, Stephan Bosch wrote:> Op 17-10-2016 om 18:05 schreef Alexander 'Leo' Bergolth: >> Does the duplicate sieve plugin do any locking to avoid duplicate >> parallel delivery of the same message?[...]>> Is there an easy way to serialize mail delivery using some locking >> inside sieve? > > We've seen this before I think. It would require some changes to the > duplicate tracking system. I'd expect the vacation command to be > affected as well.Would be great! :-)>> Or do I have to serialize per-user dovecot-lda delivery? Any experiences >> with that? > > Very little. I know there is a new lmtp_user_concurrency_limit setting, > but there is not much documentation apart from the commit message: > https://github.com/dovecot/core/commit/42abccd9b2a5a4190bd3c14ec2dcc10d51c0f491I am currently using dovecot-lda as mailbox_command, so this is not an option right now.> There are possibilities from within the MTA as well I expect.As a temporary workaround, I wrapped dovecot-lda with flock to serialize delivery: -------------------- 8< -------------------- #!/bin/sh exec /usr/bin/flock "$HOME/Maildir/INBOX" \ /usr/libexec/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT" -------------------- 8< -------------------- Cheers, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria