I spent some time setting up Dovecot LMTP with Pigeonhole this weekend.
I got most of the things I wanted to working but have a few issues that
I'm hoping that someone might be able to help with.
First, I'm running the stock version of the tools from the core repos
for CentOS/6, which means Dovecot 2.0.9 and Pigeonhole 0.2.2.
Unfortunately, I don't expect to be able to upgrade those anytime soon,
so I'm hoping I can find workarounds for my existing versions.
My mailserver is sendmail, I linked to Dovecot LMTP by adding:
FEATURE(`local_lmtp', `[IPC]', `FILE /var/run/dovecot/lmtp')dnl
to my sendmail.mc.
In my prior procmail setup, I auto-forwarded all my SPAM E-mail to a
separate user on the system with their own account (called spamuser). In
the Pigeonhole setup it looks like I have to redirect those messages
(i.e., redirect "spamuser at mydomain.com";). That works, but I'm
having
issues with "duplicate" messages getting delivered directly to my
inbox
instead of getting redirected. IOW, my regular user's .dovecot.sieve has:
if header :contains "X-Spam-Flag" "YES" {
redirect "spamuser at mydomain.com";
stop;
}
and the spamuser's .dovecot.sieve has:
require ["imap4flags"];
setflag "\\seen";
(this system is entirely for personal use, no concerns with a user's
Spam being seen via the separate spamuser account).
Now, the problem I'm having is that sometimes a message arrives with a
duplicate message-ID. Here's an abbreviated example from my maillog:
Feb 15 00:30:46 myhost sendmail[26844]: u1F5UeBP026844:
from=<ciuehazt at autoonecareers.com>, size=613, class=0, nrcpts=1,
msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at
hverovvxzb.arkansas-email.com>,
proto=SMTP, daemon=MTA,
relay=117.27.191.61.broad.static.hf.ah.cndata.com [61.191.27.117] (may
be forged)
Feb 15 00:31:05 myhost dovecot: lmtp(26856): Connect from local
Feb 15 00:31:07 myhost sendmail[26867]: u1F5UvFi026867:
from=<ciuehazt at autoonecareers.com>, size=613, class=0, nrcpts=1,
msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at
hverovvxzb.arkansas-email.com>,
proto=SMTP, daemon=MTA,
relay=117.27.191.61.broad.static.hf.ah.cndata.com [61.191.27.117] (may
be forged)
Feb 15 00:31:10 myhost dovecot: lmtp(26856, dmr):
7Hj/LoBiwVboaAAACXJZQA: sieve:
msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at
hverovvxzb.arkansas-email.com>:
forwarded to <spamuser at mydomain.com>
Feb 15 00:31:10 myhost dovecot: lmtp(26856): Disconnect from local:
Client quit
Feb 15 00:31:10 myhost dovecot: lmtp(26893): Connect from local
Feb 15 00:31:10 myhost dovecot: lmtp(26893, spamuser):
9TPnN55iwVYNaQAACXJZQA: sieve:
msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at
hverovvxzb.arkansas-email.com>:
stored mail into mailbox 'INBOX'
Feb 15 00:31:11 myhost dovecot: lmtp(26893): Disconnect from local:
Client quit
Feb 15 00:31:18 myhost dovecot: lmtp(26893): Connect from local
Feb 15 00:31:18 myhost dovecot: lmtp(26893, dmr):
+TPnN55iwVYNaQAACXJZQA: sieve:
msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at
hverovvxzb.arkansas-email.com>:
discarded duplicate forward to <spamuser at biblestuph.com>
Feb 15 00:31:18 myhost dovecot: lmtp(26893, dmr):
+TPnN55iwVYNaQAACXJZQA: sieve:
msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at
hverovvxzb.arkansas-email.com>:
stored mail into mailbox 'INBOX'
So the first message comes in and is redirected properly to my spamuser
and stored in the inbox. The second one comes in, pigeonhole sees it's a
duplicate, refuses to redirect it, and stores it in my inbox instead.
I tried forcing the .dovecot.lda-dupes file to be a symlink to /dev/null
just to see if it would work, but unfortunately dovecot just recreates
it as a normal file the next time it delivers to that user.
Ideally, I'd like to just discard the duplicates. It looks like there is
duplicate testing functionality available, but not until later versions
of Dovecot/Pigeonhole.
Is there anyway I can either just discard the duplicates, or get them to
be redirected to the spamuser?
Also, some other questions I came up with along the way:
Is there any way I can force a message to fileinto a different user's
Mailbox? I'm guessing no since it appears that the lmtp drops root
privileges before the global sieve script is interpreted, but thought
I'd ask anyway. I did try:
fileinto "/var/mail/spamuser";
But Dovecot complained, something about the mailbox pattern being
invalid. If it were possible, I would think it would want something like
fileinto "spamuser:INBOX";
instead, but I don't know if that is even possible.
In a global sieve script, is there anyway to know/test which system user
is the targeted user for delivery? So in a global sieve_before script I
could test the target user, and if it's spamuser just file it
immediately and stop with no further testing?
Thanks for any assistance!
Dan
Op 2/16/2016 om 1:32 AM schreef Dan Ragle:> In my prior procmail setup, I auto-forwarded all my SPAM E-mail to a > separate user on the system with their own account (called spamuser). > In the Pigeonhole setup it looks like I have to redirect those > messages (i.e., redirect "spamuser at mydomain.com";). That works, but > I'm having issues with "duplicate" messages getting delivered directly > to my inbox instead of getting redirected. IOW, my regular user's > .dovecot.sieve has: > > if header :contains "X-Spam-Flag" "YES" { > redirect "spamuser at mydomain.com"; > stop; > } > > and the spamuser's .dovecot.sieve has: > > require ["imap4flags"]; > setflag "\\seen"; > > (this system is entirely for personal use, no concerns with a user's > Spam being seen via the separate spamuser account). > > Now, the problem I'm having is that sometimes a message arrives with a > duplicate message-ID. Here's an abbreviated example from my maillog: > > Feb 15 00:30:46 myhost sendmail[26844]: u1F5UeBP026844: > from=<ciuehazt at autoonecareers.com>, size=613, class=0, nrcpts=1, > msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>, > proto=SMTP, daemon=MTA, > relay=117.27.191.61.broad.static.hf.ah.cndata.com [61.191.27.117] (may > be forged) > Feb 15 00:31:05 myhost dovecot: lmtp(26856): Connect from local > Feb 15 00:31:07 myhost sendmail[26867]: u1F5UvFi026867: > from=<ciuehazt at autoonecareers.com>, size=613, class=0, nrcpts=1, > msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>, > proto=SMTP, daemon=MTA, > relay=117.27.191.61.broad.static.hf.ah.cndata.com [61.191.27.117] (may > be forged) > Feb 15 00:31:10 myhost dovecot: lmtp(26856, dmr): > 7Hj/LoBiwVboaAAACXJZQA: sieve: > msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>: > forwarded to <spamuser at mydomain.com> > Feb 15 00:31:10 myhost dovecot: lmtp(26856): Disconnect from local: > Client quit > Feb 15 00:31:10 myhost dovecot: lmtp(26893): Connect from local > Feb 15 00:31:10 myhost dovecot: lmtp(26893, spamuser): > 9TPnN55iwVYNaQAACXJZQA: sieve: > msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>: > stored mail into mailbox 'INBOX' > Feb 15 00:31:11 myhost dovecot: lmtp(26893): Disconnect from local: > Client quit > Feb 15 00:31:18 myhost dovecot: lmtp(26893): Connect from local > Feb 15 00:31:18 myhost dovecot: lmtp(26893, dmr): > +TPnN55iwVYNaQAACXJZQA: sieve: > msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>: > discarded duplicate forward to <spamuser at biblestuph.com> > Feb 15 00:31:18 myhost dovecot: lmtp(26893, dmr): > +TPnN55iwVYNaQAACXJZQA: sieve: > msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>: > stored mail into mailbox 'INBOX' > > So the first message comes in and is redirected properly to my > spamuser and stored in the inbox. The second one comes in, pigeonhole > sees it's a duplicate, refuses to redirect it, and stores it in my > inbox instead.This expected behavior.> I tried forcing the .dovecot.lda-dupes file to be a symlink to > /dev/null just to see if it would work, but unfortunately dovecot just > recreates it as a normal file the next time it delivers to that user.Exactly.> Ideally, I'd like to just discard the duplicates. It looks like there > is duplicate testing functionality available, but not until later > versions of Dovecot/Pigeonhole.Yes, but you will not need that.> Is there anyway I can either just discard the duplicates, or get them > to be redirected to the spamuser?What happens is that the redirect action is ignored the second time, which means that the implicit keep is not canceled (https://tools.ietf.org/html/rfc5228#section-2.10.2). Upon executing "stop;", the script ends and the implicit keep is executed, hence the message is stored in "INBOX". So, what you need to do is cancel the implicit keep, no matter what redirect does. This can be achieved as follows: if header :contains "X-Spam-Flag" "YES" { redirect "spamuser at mydomain.com"; discard; stop; } The discard action will cancel the implicit keep. It will not affect the redirect action in any way. If you're a bit scared of the discard action, you can also replace the it with some other action that cancels the implicit keep, such as "fileinto" to put duplicates in their own little black hole folder.> Also, some other questions I came up with along the way: > > Is there any way I can force a message to fileinto a different user's > Mailbox? I'm guessing no since it appears that the lmtp drops root > privileges before the global sieve script is interpreted, but thought > I'd ask anyway. I did try: > > fileinto "/var/mail/spamuser"; > > But Dovecot complained, something about the mailbox pattern being > invalid. If it were possible, I would think it would want something like > > fileinto "spamuser:INBOX"; > > instead, but I don't know if that is even possible.Your guess is right on the money.> In a global sieve script, is there anyway to know/test which system > user is the targeted user for delivery? So in a global sieve_before > script I could test the target user, and if it's spamuser just file it > immediately and stop with no further testing?The latest versions support this: https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-dovecot-environment.txt For older versions, you can use the non-standard well-hidden "auth" field for the envelope test. This is some deprecated heritage from the old CMU implementation. require ["fileinto", "envelope"]; if envelope "auth" "spamuser" { fileinto "INBOX"; stop; } Regards, Stephan.
Perfect! Using the "auth" trick and appropriate "discard" statements allowed me to get the scripts setup exactly as I had originally intended. Thanks! Dan On 2/15/2016 8:29 PM, Stephan Bosch wrote:> Op 2/16/2016 om 1:32 AM schreef Dan Ragle: >> In my prior procmail setup, I auto-forwarded all my SPAM E-mail to a >> separate user on the system with their own account (called spamuser). >> In the Pigeonhole setup it looks like I have to redirect those >> messages (i.e., redirect "spamuser at mydomain.com";). That works, but >> I'm having issues with "duplicate" messages getting delivered directly >> to my inbox instead of getting redirected. IOW, my regular user's >> .dovecot.sieve has: >> >> if header :contains "X-Spam-Flag" "YES" { >> redirect "spamuser at mydomain.com"; >> stop; >> } >> >> and the spamuser's .dovecot.sieve has: >> >> require ["imap4flags"]; >> setflag "\\seen"; >> >> (this system is entirely for personal use, no concerns with a user's >> Spam being seen via the separate spamuser account). >> >> Now, the problem I'm having is that sometimes a message arrives with a >> duplicate message-ID. Here's an abbreviated example from my maillog: >> >> Feb 15 00:30:46 myhost sendmail[26844]: u1F5UeBP026844: >> from=<ciuehazt at autoonecareers.com>, size=613, class=0, nrcpts=1, >> msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>, >> proto=SMTP, daemon=MTA, >> relay=117.27.191.61.broad.static.hf.ah.cndata.com [61.191.27.117] (may >> be forged) >> Feb 15 00:31:05 myhost dovecot: lmtp(26856): Connect from local >> Feb 15 00:31:07 myhost sendmail[26867]: u1F5UvFi026867: >> from=<ciuehazt at autoonecareers.com>, size=613, class=0, nrcpts=1, >> msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>, >> proto=SMTP, daemon=MTA, >> relay=117.27.191.61.broad.static.hf.ah.cndata.com [61.191.27.117] (may >> be forged) >> Feb 15 00:31:10 myhost dovecot: lmtp(26856, dmr): >> 7Hj/LoBiwVboaAAACXJZQA: sieve: >> msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>: >> forwarded to <spamuser at mydomain.com> >> Feb 15 00:31:10 myhost dovecot: lmtp(26856): Disconnect from local: >> Client quit >> Feb 15 00:31:10 myhost dovecot: lmtp(26893): Connect from local >> Feb 15 00:31:10 myhost dovecot: lmtp(26893, spamuser): >> 9TPnN55iwVYNaQAACXJZQA: sieve: >> msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>: >> stored mail into mailbox 'INBOX' >> Feb 15 00:31:11 myhost dovecot: lmtp(26893): Disconnect from local: >> Client quit >> Feb 15 00:31:18 myhost dovecot: lmtp(26893): Connect from local >> Feb 15 00:31:18 myhost dovecot: lmtp(26893, dmr): >> +TPnN55iwVYNaQAACXJZQA: sieve: >> msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>: >> discarded duplicate forward to <spamuser at biblestuph.com> >> Feb 15 00:31:18 myhost dovecot: lmtp(26893, dmr): >> +TPnN55iwVYNaQAACXJZQA: sieve: >> msgid=<68895654496651-HLTYGGRLPDJDVTZRHFIKGAON at hverovvxzb.arkansas-email.com>: >> stored mail into mailbox 'INBOX' >> >> So the first message comes in and is redirected properly to my >> spamuser and stored in the inbox. The second one comes in, pigeonhole >> sees it's a duplicate, refuses to redirect it, and stores it in my >> inbox instead. > > This expected behavior. > >> I tried forcing the .dovecot.lda-dupes file to be a symlink to >> /dev/null just to see if it would work, but unfortunately dovecot just >> recreates it as a normal file the next time it delivers to that user. > > Exactly. > >> Ideally, I'd like to just discard the duplicates. It looks like there >> is duplicate testing functionality available, but not until later >> versions of Dovecot/Pigeonhole. > > Yes, but you will not need that. > >> Is there anyway I can either just discard the duplicates, or get them >> to be redirected to the spamuser? > > What happens is that the redirect action is ignored the second time, > which means that the implicit keep is not canceled > (https://tools.ietf.org/html/rfc5228#section-2.10.2). Upon executing > "stop;", the script ends and the implicit keep is executed, hence the > message is stored in "INBOX". > > So, what you need to do is cancel the implicit keep, no matter what > redirect does. This can be achieved as follows: > > if header :contains "X-Spam-Flag" "YES" { > redirect "spamuser at mydomain.com"; > discard; > stop; > } > > The discard action will cancel the implicit keep. It will not affect the > redirect action in any way. If you're a bit scared of the discard > action, you can also replace the it with some other action that cancels > the implicit keep, such as "fileinto" to put duplicates in their own > little black hole folder. > >> Also, some other questions I came up with along the way: >> >> Is there any way I can force a message to fileinto a different user's >> Mailbox? I'm guessing no since it appears that the lmtp drops root >> privileges before the global sieve script is interpreted, but thought >> I'd ask anyway. I did try: >> >> fileinto "/var/mail/spamuser"; >> >> But Dovecot complained, something about the mailbox pattern being >> invalid. If it were possible, I would think it would want something like >> >> fileinto "spamuser:INBOX"; >> >> instead, but I don't know if that is even possible. > > Your guess is right on the money. > >> In a global sieve script, is there anyway to know/test which system >> user is the targeted user for delivery? So in a global sieve_before >> script I could test the target user, and if it's spamuser just file it >> immediately and stop with no further testing? > > The latest versions support this: > > https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-dovecot-environment.txt > > For older versions, you can use the non-standard well-hidden "auth" > field for the envelope test. This is some deprecated heritage from the > old CMU implementation. > > require ["fileinto", "envelope"]; > if envelope "auth" "spamuser" { > fileinto "INBOX"; > stop; > } > > Regards, > > Stephan. > > > >