Dr. Rolf Jansen
2023-Feb-10 12:18 UTC
Pigeonhole Sieve Vacation Reply-To peculiarity with inbound AWS-SES
> Am 08.02.2023 um 20:03 schrieb Michael Peddemors <michael at linuxmagic.com>: > > Dovecot vacation message issues.. > Tough for any system to do correctly. > >> The problem here is that inbound mails from third parties utilizing AWS-SES come in with an unpersonalized envelope address and SES takes returns to this as bounce messages and changes the body's From: to ?MAILER-DAEMON at xx-zzzz-1.amazonses.com?, which is not even our MAILER-DAEMON but the one of the receiver of our reply. So the receiver gets no chance to know from the headers the identity of whom replied - he may assume it from the context the actual message, though. > > We addressed this by NOT returning vacation messages to systems that don't use 'proper' values in the MAIL FROM.. Eg Mailing Lists, Sender Rewrite schemes, and a slurry of other rules.Who is we? Your organization or the Pigeonhole developers? Actually, the question is, whether this is addressed somewhere in Pigeonhole?s code already?> But the problem is that if you are using the header From, or Reply-To etc, it's too easy to be sending to forged email addresses. > > Vacation bombing attacks for instance..You got a point here, and of course I want to prevent this.> Now, there are legitimate cases of the MAIL FROM and header from not aligning, so it is best to send to the MAIL FROM addresses.. IF you don't send it to certain MAIL FROM formats, usually by not responding to anything with mailing list identifiers, auto-suppress headers, and a few others, you only end up with clean MAIL FROM to respond to.From the point of the view of our industrial customers, who are operating processes with our chemicals, this consideration is irrelevant. If they inform a production issue by mail to the responsible service technician, they expect an immediate response, since a production stop is unacceptable. OoO notices play a role here, because we would inform alternative addresses and fone numbers for attending the support case. That said, with Pigeonhole, we are almost there.> But if you have an example that is particularly bothering you, and represents your problem, we can walk through that as an example.I send an email from an account of a mail server (Postfix/Dovecot - outbound relay SES) running on an AWS-EC2 instance in S?o Paulo (Brazil) to another mail address of mine of a mail server (Postfix/Dovecot direct MX) on an AWS-EC2 instance in Frankfurt Germany, and here the Pigeonhole?s vacation reply is activated. In the following I changed my real mail address in Brazil to rolf at example.br and the real one in Germany to rolf at example.de: The Point of view of the both involved Postfixes of said transactions are: Sender (Brazil): postfix/submission/smtpd[29165]: 97006638E8: client=201-68-62-42.dsl.telesp.net.br[201.68.62.42], sasl_method=CRAM-MD5, sasl_username=rolf at example.br postfix/cleanup[29182]: 97006638E8: message-id=<707A1777-8F09-4335-99BA-70C0367C129A at example.br> postfix/qmgr[2058]: 97006638E8: from=<rolf at example.br>, size=39626, nrcpt=1 (queue active) postfix/smtp[29183]: 97006638E8: to=<rolf at example.de>, relay=email-smtp.sa-east-1.amazonaws.com[52.67.192.29]:587, delay=0.37, delays=0.05/0.01/0.13/0.18, dsn=2.0.0, status=sent (250 Ok 010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000) Receiver (Germany): postfix/smtpd[86956]: connect from d215-2.smtp-out.sa-east-1.amazonses.com[23.249.215.2] postfix/smtpd[86956]: A44AB676E3: client=d215-2.smtp-out.sa-east-1.amazonses.com[23.249.215.2] postfix/cleanup[86957]: A44AB676E3: message-id=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> postfix/qmgr[915]: A44AB676E3: from=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com>, size=40862, nrcpt=1 (queue active) postfix/local[86958]: A44AB676E3: passing <rolf at example.de> to transport=lmtp postfix/lmtp[86959]: A44AB676E3: to=<rolf at example.de>, relay=mail.example.de[private/dovecot-lmtp], delay=0.94, delays=0.83/0.01/0.04/0.07, dsn=2.0.0, status=sent (250 2.0.0 <rolf at example.de> +/goFGgl5mOwUwEAEYr/fg Saved) Sender of the OoO (Germany): postfix/qmgr[915]: 60242676F0: from=<rolf at example.de>, size=1177, nrcpt=1 (queue active) postfix/cleanup[86957]: 60242676F0: message-id=<dovecot-sieve-1676027240-343352-0 at example.de> postfix/pickup[62932]: 60242676F0: uid=xxxx from=<rolf at example.de> postfix/smtp[86963]: 60242676F0: to=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com>, relay=email-smtp.eu-central-1.amazonaws.com[3.74.180.161]:587, delay=0.31, delays=0.01/0.02/0.13/0.15, dsn=2.0.0, status=sent (250 Ok 010701863b022070-bb3e4541-8306-4438-b649-8879ec8ff666-000000) Receiver of the OoO notice (Brazil): postfix/smtpd[29184]: connect from d215-244.smtp-out.sa-east-1.amazonses.com[23.249.215.244] postfix/smtpd[29184]: 5F1346394E: client=d215-244.smtp-out.sa-east-1.amazonses.com[23.249.215.244] postfix/cleanup[29182]: 5F1346394E: message-id=<010301863b022c93-9842f45a-85ec-419f-8199-00e027888394-000000 at sa-east-1.amazonses.com> postfix/qmgr[2058]: 5F1346394E: from=<>, size=3150, nrcpt=1 (queue active) postfix/local[29185]: 5F1346394E: passing <rolf at example.br> to transport=lmtp postfix/lmtp[29186]: 5F1346394E: to=<rolf at example.br>, relay=mail.example.br[private/dovecot-lmtp], delay=0.05, delays=0.01/0.01/0.02/0.01, dsn=2.0.0, status=sent (250 2.0.0 <rolf at example.br> RRZHGWwl5mMDcgAAjZNhtw Saved) The headers of the received OoO message (I removed the DKIM stuff for clarity) are: Return-Path: <> Delivered-To: rolf at example.br Received: from mail.example.br by example.br with LMTP id RRZHGWwl5mMDcgAAjZNhtw (envelope-from <>) for <rolf at example.br>; Fri, 10 Feb 2023 08:07:24 -0300 Received: from d215-244.smtp-out.sa-east-1.amazonses.com (d215-244.smtp-out.sa-east-1.amazonses.com [23.249.215.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.example.br (Postfix) with ESMTPS id 5F1346394E for <rolf at example.br>; Fri, 10 Feb 2023 08:07:24 -0300 (-03) DKIM-Signature: ... X-Sieve: Pigeonhole Sieve 0.5.20 (149edcf2) Message-ID: <010301863b022c93-9842f45a-85ec-419f-8199-00e027888394-000000 at sa-east-1.amazonses.com> Date: Fri, 10 Feb 2023 11:07:23 +0000 From: MAILER-DAEMON at sa-east-1.amazonses.com To: rolf at example.br Subject: Out of Office - automatic notice In-Reply-To: <010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> References: <010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> Auto-Submitted: auto-replied (vacation) Precedence: bulk X-Auto-Response-Suppress: All MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Feedback-ID: 1.sa-east-1.E7DnIghDRHBvwjrx2QL73gyWAj+NYISxiq7bqOVD27E=:AmazonSES X-SES-Outgoing: 2023.02.10-23.249.215.244 Note how there is not a single reference of the real origin of the OoO message, rolf at example.de. Instead we see From: MAILER-DAEMON at sa-east-1.amazonses.com. The receiver of this message need to guess the real address from the outside context. The actual problem is that Pigeonhole responds to to=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> instead of to=<rolf at example.de>. Amazon SES of the original sender (not our relay) takes this as a bounce and changes the relevant headers. This is not under our control. Perhaps this could be mitigated by adding a Reply-To: header which informs the original sender. Then the receiver of the OoO noice would at least have a reference to what and whom this is about. Best regards Rolf> On 2023-02-08 13:21, Dr. Rolf Jansen wrote: >> Am 08.02.2023 um 12:57 schrieb Michael Peddemors <michael at linuxmagic.com>: >>> On 2023-02-08 07:47, Dr. Rolf Jansen wrote: >>>> Am 08.02.2023 um 12:23 schrieb Michael Peddemors <michael at linuxmagic.com>: >>>>> On 2023-02-07 14:57, Dr. Rolf Jansen wrote: >>>>>> Am 07.02.2023 um 19:49 schrieb Michael Peddemors <michael at linuxmagic.com>: >>>>>>> On 2023-02-07 13:33, jeremy ardley wrote: >>>>>>>> On 8/2/23 05:08, Dr. Rolf Jansen wrote: >>>>>>>>> Am 07.02.2023 um 17:54 schrieb jeremy ardley<jeremy at ardley.org>: >>>>>>>>>> On 7/2/23 22:01, Dr. Rolf Jansen wrote: >>>>>>>>>> To begin with, usage of Amazons Simple Email Service (SES) is mandatory for outgoing mails from AWS-EC2 instances. >>>>>>>>>> I run AWS-EC2 instances using postfix to send a receive mail. They can send direct assuming I set up suitable SPF, but they typically forward mail to another host under my control that is not on AWS to use as the outgoing server. >>>>>>>>> OK, that?s another use case. Many do use a full fledged Postfix/Dovecot installation. However the outgoing port 25 into the internet is blocked by AWS, and therefore we may either use a third party relay for our outgoing emails or may use SES, which is not that bad - except some unusual peculiarities. >>>>>>>>> >>>>>>>> This is off topic, but to be precise: >>>>>>>> - AWS throttles but does not block traffic to a *destination* port 25. >>>>>>>> - The *origin* port on the EC2 instance is an unprivilged port, not port 25 >>>>>>>> - If you use a relayhost you typically send from an unprivilged EC2 port to port 587 on the relay host >>>>>>>> Jeremy >>>>>>> >>>>>>> And if you DO intend to send out to port 25, remember to update the PTR on your EC2 instance. >>>>>> I respond off-list. Generally a good hint but it?s like trying to bath salts (https://en.wikipedia.org/wiki/Bath_salts) when you are taking a shower. >>>>>> https://aws.amazon.com/premiumsupport/knowledge-center/ec2-port-25-throttle/ >>>>>> Although the link text (ec2-port-25-throttle) suggests throttling, it is actually blocking - read the text. This cannot be removed from standard EC2 instances. >>>>>> This also corresponds to my experience. On the console of an AWS-EC2 instance, I entered just now: >>>>>> telnet cyclaero.com 25 >>>>>> This hangs until timeout. At the same time cyclaero.com received emails from the internet on port 25. >>>>>> Trying 3.126.110.101... >>>>>> telnet: connect to address 3.126.110.101: Operation timed out >>>>>> telnet: Unable to connect to remote host >>>>> >>>>> There are a lot of malicous or compromised EC2 instances..It is good they block port by default, and the one day delay before unblocking (you can request it) helps stop those short term spam instances. >>>>> >>>>> Now, wish they blocked port 587 by default ;) >>>>> >>>>> Or at least tracked the attacks. >>>>> >>>>> Strange though, that people still try to set up email on EC2, doesn't make sense, and it isn't the cheapest alternative.
Paul Kudla
2023-Feb-10 13:30 UTC
Pigeonhole Sieve Vacation Reply-To peculiarity with inbound AWS-SES
Good morning, I have been following this post for a bit and would like to share experience please and thanks. This is not meant to give a solution but save some massive frustration with other system as i have gone through the same issues overall. In general I found found over the past few years all the big boys are forcing all the private systems into standards that are not really defined and get implemented willy nilly. Just because microsoft starts a standard, then google picks up on it then AWS and then yahoo etc etc in any order does not mean its a proper approach. That being said is there any reason why you are not sending the emails directly yourself, ie why are you using a proxy. I found (for example) when forwarding an email from @scom.ca to gmail for example all the headers, dkim, spf records are all passed along which resulted in emails never being allowed to be delivered. Although this may be your issue directly or indirectly what i found is to forward to a gmail.com account i had to program the gmail.com account to pop my server. This does work well but only for gmail.com I have other customers where i try to pop the email from whatever system (which does work) but when i forward to an account on my system postfix rewrite the header from address to the mailxx.scom.ca email server name being used to forward the email which generates the same issues you are having in the headers being rewritten not showing the from address? My server's are setup with custom python programming filters developed over ten years and i can not seem to control anything either? I get you do production stuff (so do my customers) which is why it might be better to send via a postfix instance that you are in control of of couse this does require a static ip etc which i dont know if you have access to or not? but i think this would save a lot of frustration trying to be "COMPATIBLE" with everyone else out there that do not even follow their own standards? Just though i would pass this info along, trying to help ? Happy Friday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main?1.866.411.7266 Fax?1.888.892.7266 Email?paul at scom.ca On 2/10/2023 7:18 AM, Dr. Rolf Jansen wrote:> >> Am 08.02.2023 um 20:03 schrieb Michael Peddemors <michael at linuxmagic.com>: >> >> Dovecot vacation message issues.. >> Tough for any system to do correctly. >> >>> The problem here is that inbound mails from third parties utilizing AWS-SES come in with an unpersonalized envelope address and SES takes returns to this as bounce messages and changes the body's From: to ?MAILER-DAEMON at xx-zzzz-1.amazonses.com?, which is not even our MAILER-DAEMON but the one of the receiver of our reply. So the receiver gets no chance to know from the headers the identity of whom replied - he may assume it from the context the actual message, though. >> >> We addressed this by NOT returning vacation messages to systems that don't use 'proper' values in the MAIL FROM.. Eg Mailing Lists, Sender Rewrite schemes, and a slurry of other rules. > > Who is we? Your organization or the Pigeonhole developers? Actually, the question is, whether this is addressed somewhere in Pigeonhole?s code already? > >> But the problem is that if you are using the header From, or Reply-To etc, it's too easy to be sending to forged email addresses. >> >> Vacation bombing attacks for instance.. > > You got a point here, and of course I want to prevent this. > >> Now, there are legitimate cases of the MAIL FROM and header from not aligning, so it is best to send to the MAIL FROM addresses.. IF you don't send it to certain MAIL FROM formats, usually by not responding to anything with mailing list identifiers, auto-suppress headers, and a few others, you only end up with clean MAIL FROM to respond to. > > From the point of the view of our industrial customers, who are operating processes with our chemicals, this consideration is irrelevant. If they inform a production issue by mail to the responsible service technician, they expect an immediate response, since a production stop is unacceptable. OoO notices play a role here, because we would inform alternative addresses and fone numbers for attending the support case. > > That said, with Pigeonhole, we are almost there. > >> But if you have an example that is particularly bothering you, and represents your problem, we can walk through that as an example. > > I send an email from an account of a mail server (Postfix/Dovecot - outbound relay SES) running on an AWS-EC2 instance in S?o Paulo (Brazil) to another mail address of mine of a mail server (Postfix/Dovecot direct MX) on an AWS-EC2 instance in Frankfurt Germany, and here the Pigeonhole?s vacation reply is activated. > > In the following I changed my real mail address in Brazil to rolf at example.br and the real one in Germany to rolf at example.de: > > The Point of view of the both involved Postfixes of said transactions are: > > Sender (Brazil): > postfix/submission/smtpd[29165]: 97006638E8: client=201-68-62-42.dsl.telesp.net.br[201.68.62.42], sasl_method=CRAM-MD5, sasl_username=rolf at example.br > postfix/cleanup[29182]: 97006638E8: message-id=<707A1777-8F09-4335-99BA-70C0367C129A at example.br> > postfix/qmgr[2058]: 97006638E8: from=<rolf at example.br>, size=39626, nrcpt=1 (queue active) > postfix/smtp[29183]: 97006638E8: to=<rolf at example.de>, relay=email-smtp.sa-east-1.amazonaws.com[52.67.192.29]:587, delay=0.37, delays=0.05/0.01/0.13/0.18, dsn=2.0.0, status=sent (250 Ok 010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000) > > Receiver (Germany): > postfix/smtpd[86956]: connect from d215-2.smtp-out.sa-east-1.amazonses.com[23.249.215.2] > postfix/smtpd[86956]: A44AB676E3: client=d215-2.smtp-out.sa-east-1.amazonses.com[23.249.215.2] > postfix/cleanup[86957]: A44AB676E3: message-id=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> > postfix/qmgr[915]: A44AB676E3: from=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com>, size=40862, nrcpt=1 (queue active) > postfix/local[86958]: A44AB676E3: passing <rolf at example.de> to transport=lmtp > postfix/lmtp[86959]: A44AB676E3: to=<rolf at example.de>, relay=mail.example.de[private/dovecot-lmtp], delay=0.94, delays=0.83/0.01/0.04/0.07, dsn=2.0.0, status=sent (250 2.0.0 <rolf at example.de> +/goFGgl5mOwUwEAEYr/fg Saved) > > Sender of the OoO (Germany): > postfix/qmgr[915]: 60242676F0: from=<rolf at example.de>, size=1177, nrcpt=1 (queue active) > postfix/cleanup[86957]: 60242676F0: message-id=<dovecot-sieve-1676027240-343352-0 at example.de> > postfix/pickup[62932]: 60242676F0: uid=xxxx from=<rolf at example.de> > postfix/smtp[86963]: 60242676F0: to=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com>, relay=email-smtp.eu-central-1.amazonaws.com[3.74.180.161]:587, delay=0.31, delays=0.01/0.02/0.13/0.15, dsn=2.0.0, status=sent (250 Ok 010701863b022070-bb3e4541-8306-4438-b649-8879ec8ff666-000000) > > Receiver of the OoO notice (Brazil): > postfix/smtpd[29184]: connect from d215-244.smtp-out.sa-east-1.amazonses.com[23.249.215.244] > postfix/smtpd[29184]: 5F1346394E: client=d215-244.smtp-out.sa-east-1.amazonses.com[23.249.215.244] > postfix/cleanup[29182]: 5F1346394E: message-id=<010301863b022c93-9842f45a-85ec-419f-8199-00e027888394-000000 at sa-east-1.amazonses.com> > postfix/qmgr[2058]: 5F1346394E: from=<>, size=3150, nrcpt=1 (queue active) > postfix/local[29185]: 5F1346394E: passing <rolf at example.br> to transport=lmtp > postfix/lmtp[29186]: 5F1346394E: to=<rolf at example.br>, relay=mail.example.br[private/dovecot-lmtp], delay=0.05, delays=0.01/0.01/0.02/0.01, dsn=2.0.0, status=sent (250 2.0.0 <rolf at example.br> RRZHGWwl5mMDcgAAjZNhtw Saved) > > The headers of the received OoO message (I removed the DKIM stuff for clarity) are: > > Return-Path: <> > Delivered-To: rolf at example.br > Received: from mail.example.br > by example.br with LMTP > id RRZHGWwl5mMDcgAAjZNhtw > (envelope-from <>) > for <rolf at example.br>; Fri, 10 Feb 2023 08:07:24 -0300 > Received: from d215-244.smtp-out.sa-east-1.amazonses.com (d215-244.smtp-out.sa-east-1.amazonses.com [23.249.215.244]) > (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) > (No client certificate requested) > by mail.example.br (Postfix) with ESMTPS id 5F1346394E > for <rolf at example.br>; Fri, 10 Feb 2023 08:07:24 -0300 (-03) > DKIM-Signature: ... > X-Sieve: Pigeonhole Sieve 0.5.20 (149edcf2) > Message-ID: <010301863b022c93-9842f45a-85ec-419f-8199-00e027888394-000000 at sa-east-1.amazonses.com> > Date: Fri, 10 Feb 2023 11:07:23 +0000 > From: MAILER-DAEMON at sa-east-1.amazonses.com > To: rolf at example.br > Subject: Out of Office - automatic notice > In-Reply-To: <010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> > References: <010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> > Auto-Submitted: auto-replied (vacation) > Precedence: bulk > X-Auto-Response-Suppress: All > MIME-Version: 1.0 > Content-Type: text/plain; charset=utf-8 > Content-Transfer-Encoding: 8bit > Feedback-ID: 1.sa-east-1.E7DnIghDRHBvwjrx2QL73gyWAj+NYISxiq7bqOVD27E=:AmazonSES > X-SES-Outgoing: 2023.02.10-23.249.215.244 > > > Note how there is not a single reference of the real origin of the OoO message, rolf at example.de. Instead we see From: MAILER-DAEMON at sa-east-1.amazonses.com. The receiver of this message need to guess the real address from the outside context. > > The actual problem is that Pigeonhole responds to to=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> instead of to=<rolf at example.de>. Amazon SES of the original sender (not our relay) takes this as a bounce and changes the relevant headers. This is not under our control. > > Perhaps this could be mitigated by adding a Reply-To: header which informs the original sender. Then the receiver of the OoO noice would at least have a reference to what and whom this is about. > > Best regards > > Rolf > >> On 2023-02-08 13:21, Dr. Rolf Jansen wrote: >>> Am 08.02.2023 um 12:57 schrieb Michael Peddemors <michael at linuxmagic.com>: >>>> On 2023-02-08 07:47, Dr. Rolf Jansen wrote: >>>>> Am 08.02.2023 um 12:23 schrieb Michael Peddemors <michael at linuxmagic.com>: >>>>>> On 2023-02-07 14:57, Dr. Rolf Jansen wrote: >>>>>>> Am 07.02.2023 um 19:49 schrieb Michael Peddemors <michael at linuxmagic.com>: >>>>>>>> On 2023-02-07 13:33, jeremy ardley wrote: >>>>>>>>> On 8/2/23 05:08, Dr. Rolf Jansen wrote: >>>>>>>>>> Am 07.02.2023 um 17:54 schrieb jeremy ardley<jeremy at ardley.org>: >>>>>>>>>>> On 7/2/23 22:01, Dr. Rolf Jansen wrote: >>>>>>>>>>> To begin with, usage of Amazons Simple Email Service (SES) is mandatory for outgoing mails from AWS-EC2 instances. >>>>>>>>>>> I run AWS-EC2 instances using postfix to send a receive mail. They can send direct assuming I set up suitable SPF, but they typically forward mail to another host under my control that is not on AWS to use as the outgoing server. >>>>>>>>>> OK, that?s another use case. Many do use a full fledged Postfix/Dovecot installation. However the outgoing port 25 into the internet is blocked by AWS, and therefore we may either use a third party relay for our outgoing emails or may use SES, which is not that bad - except some unusual peculiarities. >>>>>>>>>> >>>>>>>>> This is off topic, but to be precise: >>>>>>>>> - AWS throttles but does not block traffic to a *destination* port 25. >>>>>>>>> - The *origin* port on the EC2 instance is an unprivilged port, not port 25 >>>>>>>>> - If you use a relayhost you typically send from an unprivilged EC2 port to port 587 on the relay host >>>>>>>>> Jeremy >>>>>>>> >>>>>>>> And if you DO intend to send out to port 25, remember to update the PTR on your EC2 instance. >>>>>>> I respond off-list. Generally a good hint but it?s like trying to bath salts (https://en.wikipedia.org/wiki/Bath_salts) when you are taking a shower. >>>>>>> https://aws.amazon.com/premiumsupport/knowledge-center/ec2-port-25-throttle/ >>>>>>> Although the link text (ec2-port-25-throttle) suggests throttling, it is actually blocking - read the text. This cannot be removed from standard EC2 instances. >>>>>>> This also corresponds to my experience. On the console of an AWS-EC2 instance, I entered just now: >>>>>>> telnet cyclaero.com 25 >>>>>>> This hangs until timeout. At the same time cyclaero.com received emails from the internet on port 25. >>>>>>> Trying 3.126.110.101... >>>>>>> telnet: connect to address 3.126.110.101: Operation timed out >>>>>>> telnet: Unable to connect to remote host >>>>>> >>>>>> There are a lot of malicous or compromised EC2 instances..It is good they block port by default, and the one day delay before unblocking (you can request it) helps stop those short term spam instances. >>>>>> >>>>>> Now, wish they blocked port 587 by default ;) >>>>>> >>>>>> Or at least tracked the attacks. >>>>>> >>>>>> Strange though, that people still try to set up email on EC2, doesn't make sense, and it isn't the cheapest alternative. > >
Michael Peddemors
2023-Feb-10 14:51 UTC
[OFF TOPIC] Re: Pigeonhole Sieve Vacation Reply-To peculiarity with inbound AWS-SES
TOP POSTING for clarity I think this is getting off topic for the dovecot list. Vacation messaging is a complex topic, and for the record it does seem that the way they are doing vacation messages could use improvement. This should NOT be sent as a BOUNCE <>, and it should NOT come from MAILER-DAEMON, as it is actually from the person with the vacation message. It also should NOT have a precedence header of BULK. In the short term, you might need to reconsider how you handle vacation messages, in the long term you should file a bug report through the appropriate channels. The REAL problem of course stems from the MAIL FROM via SES. Return-Path: <010701863b42f48e-59d7870d-22cc-4dfe-a34f-0415ac334045-000000 at eu-central-1.amazonses.com> This is long a pet peeve, where systems don't utilize the actual sender address in the MAIL FROM. There are many things I would like to do based on the data in the MAIL FROM, but this is obfuscation for obfuscation sake.. I would expect that to be.. Return-Path: <dovecot-rj at cyclaero.com> Simply put, fix that, and you fix your problem ;) This is why vacation messages need filters, so they don't respond to mailing lists, bulk mailers, automated emails, etc. On 2023-02-10 04:18, Dr. Rolf Jansen wrote:>> Am 08.02.2023 um 20:03 schrieb Michael Peddemors <michael at linuxmagic.com>: >> >> Dovecot vacation message issues.. >> Tough for any system to do correctly. >> >>> The problem here is that inbound mails from third parties utilizing AWS-SES come in with an unpersonalized envelope address and SES takes returns to this as bounce messages and changes the body's From: to ?MAILER-DAEMON at xx-zzzz-1.amazonses.com?, which is not even our MAILER-DAEMON but the one of the receiver of our reply. So the receiver gets no chance to know from the headers the identity of whom replied - he may assume it from the context the actual message, though. >> >> We addressed this by NOT returning vacation messages to systems that don't use 'proper' values in the MAIL FROM.. Eg Mailing Lists, Sender Rewrite schemes, and a slurry of other rules. > > Who is we? Your organization or the Pigeonhole developers? Actually, the question is, whether this is addressed somewhere in Pigeonhole?s code already? > >> But the problem is that if you are using the header From, or Reply-To etc, it's too easy to be sending to forged email addresses. >> >> Vacation bombing attacks for instance.. > > You got a point here, and of course I want to prevent this. > >> Now, there are legitimate cases of the MAIL FROM and header from not aligning, so it is best to send to the MAIL FROM addresses.. IF you don't send it to certain MAIL FROM formats, usually by not responding to anything with mailing list identifiers, auto-suppress headers, and a few others, you only end up with clean MAIL FROM to respond to. > > From the point of the view of our industrial customers, who are operating processes with our chemicals, this consideration is irrelevant. If they inform a production issue by mail to the responsible service technician, they expect an immediate response, since a production stop is unacceptable. OoO notices play a role here, because we would inform alternative addresses and fone numbers for attending the support case. > > That said, with Pigeonhole, we are almost there. > >> But if you have an example that is particularly bothering you, and represents your problem, we can walk through that as an example. > > I send an email from an account of a mail server (Postfix/Dovecot - outbound relay SES) running on an AWS-EC2 instance in S?o Paulo (Brazil) to another mail address of mine of a mail server (Postfix/Dovecot direct MX) on an AWS-EC2 instance in Frankfurt Germany, and here the Pigeonhole?s vacation reply is activated. > > In the following I changed my real mail address in Brazil to rolf at example.br and the real one in Germany to rolf at example.de: > > The Point of view of the both involved Postfixes of said transactions are: > > Sender (Brazil): > postfix/submission/smtpd[29165]: 97006638E8: client=201-68-62-42.dsl.telesp.net.br[201.68.62.42], sasl_method=CRAM-MD5, sasl_username=rolf at example.br > postfix/cleanup[29182]: 97006638E8: message-id=<707A1777-8F09-4335-99BA-70C0367C129A at example.br> > postfix/qmgr[2058]: 97006638E8: from=<rolf at example.br>, size=39626, nrcpt=1 (queue active) > postfix/smtp[29183]: 97006638E8: to=<rolf at example.de>, relay=email-smtp.sa-east-1.amazonaws.com[52.67.192.29]:587, delay=0.37, delays=0.05/0.01/0.13/0.18, dsn=2.0.0, status=sent (250 Ok 010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000) > > Receiver (Germany): > postfix/smtpd[86956]: connect from d215-2.smtp-out.sa-east-1.amazonses.com[23.249.215.2] > postfix/smtpd[86956]: A44AB676E3: client=d215-2.smtp-out.sa-east-1.amazonses.com[23.249.215.2] > postfix/cleanup[86957]: A44AB676E3: message-id=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> > postfix/qmgr[915]: A44AB676E3: from=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com>, size=40862, nrcpt=1 (queue active) > postfix/local[86958]: A44AB676E3: passing <rolf at example.de> to transport=lmtp > postfix/lmtp[86959]: A44AB676E3: to=<rolf at example.de>, relay=mail.example.de[private/dovecot-lmtp], delay=0.94, delays=0.83/0.01/0.04/0.07, dsn=2.0.0, status=sent (250 2.0.0 <rolf at example.de> +/goFGgl5mOwUwEAEYr/fg Saved) > > Sender of the OoO (Germany): > postfix/qmgr[915]: 60242676F0: from=<rolf at example.de>, size=1177, nrcpt=1 (queue active) > postfix/cleanup[86957]: 60242676F0: message-id=<dovecot-sieve-1676027240-343352-0 at example.de> > postfix/pickup[62932]: 60242676F0: uid=xxxx from=<rolf at example.de> > postfix/smtp[86963]: 60242676F0: to=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com>, relay=email-smtp.eu-central-1.amazonaws.com[3.74.180.161]:587, delay=0.31, delays=0.01/0.02/0.13/0.15, dsn=2.0.0, status=sent (250 Ok 010701863b022070-bb3e4541-8306-4438-b649-8879ec8ff666-000000) > > Receiver of the OoO notice (Brazil): > postfix/smtpd[29184]: connect from d215-244.smtp-out.sa-east-1.amazonses.com[23.249.215.244] > postfix/smtpd[29184]: 5F1346394E: client=d215-244.smtp-out.sa-east-1.amazonses.com[23.249.215.244] > postfix/cleanup[29182]: 5F1346394E: message-id=<010301863b022c93-9842f45a-85ec-419f-8199-00e027888394-000000 at sa-east-1.amazonses.com> > postfix/qmgr[2058]: 5F1346394E: from=<>, size=3150, nrcpt=1 (queue active) > postfix/local[29185]: 5F1346394E: passing <rolf at example.br> to transport=lmtp > postfix/lmtp[29186]: 5F1346394E: to=<rolf at example.br>, relay=mail.example.br[private/dovecot-lmtp], delay=0.05, delays=0.01/0.01/0.02/0.01, dsn=2.0.0, status=sent (250 2.0.0 <rolf at example.br> RRZHGWwl5mMDcgAAjZNhtw Saved) > > The headers of the received OoO message (I removed the DKIM stuff for clarity) are: > > Return-Path: <> > Delivered-To: rolf at example.br > Received: from mail.example.br > by example.br with LMTP > id RRZHGWwl5mMDcgAAjZNhtw > (envelope-from <>) > for <rolf at example.br>; Fri, 10 Feb 2023 08:07:24 -0300 > Received: from d215-244.smtp-out.sa-east-1.amazonses.com (d215-244.smtp-out.sa-east-1.amazonses.com [23.249.215.244]) > (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) > (No client certificate requested) > by mail.example.br (Postfix) with ESMTPS id 5F1346394E > for <rolf at example.br>; Fri, 10 Feb 2023 08:07:24 -0300 (-03) > DKIM-Signature: ... > X-Sieve: Pigeonhole Sieve 0.5.20 (149edcf2) > Message-ID: <010301863b022c93-9842f45a-85ec-419f-8199-00e027888394-000000 at sa-east-1.amazonses.com> > Date: Fri, 10 Feb 2023 11:07:23 +0000 > From: MAILER-DAEMON at sa-east-1.amazonses.com > To: rolf at example.br > Subject: Out of Office - automatic notice > In-Reply-To: <010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> > References: <010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> > Auto-Submitted: auto-replied (vacation) > Precedence: bulk > X-Auto-Response-Suppress: All > MIME-Version: 1.0 > Content-Type: text/plain; charset=utf-8 > Content-Transfer-Encoding: 8bit > Feedback-ID: 1.sa-east-1.E7DnIghDRHBvwjrx2QL73gyWAj+NYISxiq7bqOVD27E=:AmazonSES > X-SES-Outgoing: 2023.02.10-23.249.215.244 > > > Note how there is not a single reference of the real origin of the OoO message, rolf at example.de. Instead we see From: MAILER-DAEMON at sa-east-1.amazonses.com. The receiver of this message need to guess the real address from the outside context. > > The actual problem is that Pigeonhole responds to to=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000 at sa-east-1.amazonses.com> instead of to=<rolf at example.de>. Amazon SES of the original sender (not our relay) takes this as a bounce and changes the relevant headers. This is not under our control. > > Perhaps this could be mitigated by adding a Reply-To: header which informs the original sender. Then the receiver of the OoO noice would at least have a reference to what and whom this is about. > > Best regards > > Rolf > >> On 2023-02-08 13:21, Dr. Rolf Jansen wrote: >>> Am 08.02.2023 um 12:57 schrieb Michael Peddemors <michael at linuxmagic.com>: >>>> On 2023-02-08 07:47, Dr. Rolf Jansen wrote: >>>>> Am 08.02.2023 um 12:23 schrieb Michael Peddemors <michael at linuxmagic.com>: >>>>>> On 2023-02-07 14:57, Dr. Rolf Jansen wrote: >>>>>>> Am 07.02.2023 um 19:49 schrieb Michael Peddemors <michael at linuxmagic.com>: >>>>>>>> On 2023-02-07 13:33, jeremy ardley wrote: >>>>>>>>> On 8/2/23 05:08, Dr. Rolf Jansen wrote: >>>>>>>>>> Am 07.02.2023 um 17:54 schrieb jeremy ardley<jeremy at ardley.org>: >>>>>>>>>>> On 7/2/23 22:01, Dr. Rolf Jansen wrote: >>>>>>>>>>> To begin with, usage of Amazons Simple Email Service (SES) is mandatory for outgoing mails from AWS-EC2 instances. >>>>>>>>>>> I run AWS-EC2 instances using postfix to send a receive mail. They can send direct assuming I set up suitable SPF, but they typically forward mail to another host under my control that is not on AWS to use as the outgoing server. >>>>>>>>>> OK, that?s another use case. Many do use a full fledged Postfix/Dovecot installation. However the outgoing port 25 into the internet is blocked by AWS, and therefore we may either use a third party relay for our outgoing emails or may use SES, which is not that bad - except some unusual peculiarities. >>>>>>>>>> >>>>>>>>> This is off topic, but to be precise: >>>>>>>>> - AWS throttles but does not block traffic to a *destination* port 25. >>>>>>>>> - The *origin* port on the EC2 instance is an unprivilged port, not port 25 >>>>>>>>> - If you use a relayhost you typically send from an unprivilged EC2 port to port 587 on the relay host >>>>>>>>> Jeremy >>>>>>>> >>>>>>>> And if you DO intend to send out to port 25, remember to update the PTR on your EC2 instance. >>>>>>> I respond off-list. Generally a good hint but it?s like trying to bath salts (https://en.wikipedia.org/wiki/Bath_salts) when you are taking a shower. >>>>>>> https://aws.amazon.com/premiumsupport/knowledge-center/ec2-port-25-throttle/ >>>>>>> Although the link text (ec2-port-25-throttle) suggests throttling, it is actually blocking - read the text. This cannot be removed from standard EC2 instances. >>>>>>> This also corresponds to my experience. On the console of an AWS-EC2 instance, I entered just now: >>>>>>> telnet cyclaero.com 25 >>>>>>> This hangs until timeout. At the same time cyclaero.com received emails from the internet on port 25. >>>>>>> Trying 3.126.110.101... >>>>>>> telnet: connect to address 3.126.110.101: Operation timed out >>>>>>> telnet: Unable to connect to remote host >>>>>> >>>>>> There are a lot of malicous or compromised EC2 instances..It is good they block port by default, and the one day delay before unblocking (you can request it) helps stop those short term spam instances. >>>>>> >>>>>> Now, wish they blocked port 587 by default ;) >>>>>> >>>>>> Or at least tracked the attacks. >>>>>> >>>>>> Strange though, that people still try to set up email on EC2, doesn't make sense, and it isn't the cheapest alternative. >-- "Catch the Magic of Linux..." ------------------------------------------------------------------------ Michael Peddemors, President/CEO LinuxMagic Inc. Visit us at http://www.linuxmagic.com @linuxmagic A Wizard IT Company - For More Info http://www.wizard.ca "LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd. ------------------------------------------------------------------------ 604-682-0300 Beautiful British Columbia, Canada This email and any electronic data contained are confidential and intended solely for the use of the individual or entity to which they are addressed. Please note that any views or opinions presented in this email are solely those of the author and are not intended to represent those of the company.