Hi, Several folks have let me know that my messages sent via lists.xen.org are marked as spam / spoofed, especially when using Gmail to receive Xen mail. I believe this is because outbound Amazon email contains a DKIM signature. When Mailman modifies my message and re-sends it, the DKIM signature is invalidated [1]. To work around this, Mailman 2.1.10 and later contain a configuration variable called "REMOVE_DKIM_HEADERS" [2]. Perhaps if this were turned on we''d work around the problem. Matt [1] http://wiki.list.org/display/DEV/DKIM [2] https://bugs.launchpad.net/mailman/+bug/557493
Matt Wilson writes ("[Xen-devel] lists.xen.org Mailman configuration and
DKIM"):> Several folks have let me know that my messages sent via lists.xen.org
> are marked as spam / spoofed, especially when using Gmail to receive
> Xen mail. I believe this is because outbound Amazon email contains a
> DKIM signature. When Mailman modifies my message and re-sends it, the
> DKIM signature is invalidated [1].
>
> To work around this, Mailman 2.1.10 and later contain a configuration
> variable called "REMOVE_DKIM_HEADERS" [2]. Perhaps if this were
turned
> on we''d work around the problem.
...> [1] http://wiki.list.org/display/DEV/DKIM
> [2] https://bugs.launchpad.net/mailman/+bug/557493
Having checked RFC4871 I think it is clear that according to the
standards
- Mailman SHOULD NOT [1] strip DKIM-Signature
- No-one should treat a message with an invalid DKIM signature
differently from a message with no DKIM signature at all [2]
[1] 4871 says in s3.5 that DKIM-Signature SHOULD be treated the same
way as a trace header (ie a Received), so removing it would be a
violation of that SHOULD not necessarily a violation of the MUST NOT
mess with Received headers.
[2] RFC4871 6.1:
A verifier SHOULD NOT treat a message that has one or more bad
signatures and no good signatures differently from a message with
no signature at all; such treatment is a matter of local policy and
is beyond the scope of this document.
I think it would be better if you would do one of:
(a) Get Gmail fixed to comply with RFC4871 6.1;
(b) Get your correspondents to use a non-broken email host;
(c) Get the DKIM the spec changed or clarified;
(d) Stop putting these abused things in your email headers.
That would be better than asking lists.xen.org to start violating the
specified protocol. Now of course a SHOULD is not an absolute
requirement. Perhaps mailing lists are a special case somehow; but if
so I would expect this to be addressed in the relevant standards
documents. I don''t see any particular reason to think that
lists.xen.org is somehow unusual.
Do you agree ?
Ian.
On 03/08/12 15:44, Ian Jackson wrote:> Matt Wilson writes ("[Xen-devel] lists.xen.org Mailman configuration and DKIM"): >> Several folks have let me know that my messages sent via lists.xen.org >> are marked as spam / spoofed, especially when using Gmail to receive >> Xen mail. I believe this is because outbound Amazon email contains a >> DKIM signature. When Mailman modifies my message and re-sends it, the >> DKIM signature is invalidated [1]. >> >> To work around this, Mailman 2.1.10 and later contain a configuration >> variable called "REMOVE_DKIM_HEADERS" [2]. Perhaps if this were turned >> on we''d work around the problem. > ... >> [1] http://wiki.list.org/display/DEV/DKIM >> [2] https://bugs.launchpad.net/mailman/+bug/557493 > Having checked RFC4871 I think it is clear that according to the > standards > - Mailman SHOULD NOT [1] strip DKIM-Signature > - No-one should treat a message with an invalid DKIM signature > differently from a message with no DKIM signature at all [2]It''s actually pretty likely that gmail would also reject the mail if it had no DKIM signature, isn''t it? In which case stripping the signature wouldn''t really help. -George
On Fri, Aug 03, 2012 at 07:44:30AM -0700, Ian Jackson wrote:> Matt Wilson writes ("[Xen-devel] lists.xen.org Mailman configuration and DKIM"): > > Several folks have let me know that my messages sent via lists.xen.org > > are marked as spam / spoofed, especially when using Gmail to receive > > Xen mail. I believe this is because outbound Amazon email contains a > > DKIM signature. When Mailman modifies my message and re-sends it, the > > DKIM signature is invalidated [1]. > > > > To work around this, Mailman 2.1.10 and later contain a configuration > > variable called "REMOVE_DKIM_HEADERS" [2]. Perhaps if this were turned > > on we''d work around the problem. > ... > > [1] http://wiki.list.org/display/DEV/DKIM > > [2] https://bugs.launchpad.net/mailman/+bug/557493 > > Having checked RFC4871 I think it is clear that according to the > standards > - Mailman SHOULD NOT [1] strip DKIM-Signature > - No-one should treat a message with an invalid DKIM signature > differently from a message with no DKIM signature at all [2] > > [1] 4871 says in s3.5 that DKIM-Signature SHOULD be treated the same > way as a trace header (ie a Received), so removing it would be a > violation of that SHOULD not necessarily a violation of the MUST NOT > mess with Received headers. > > [2] RFC4871 6.1: > A verifier SHOULD NOT treat a message that has one or more bad > signatures and no good signatures differently from a message with > no signature at all; such treatment is a matter of local policy and > is beyond the scope of this document.> I think it would be better if you would do one of: > (a) Get Gmail fixed to comply with RFC4871 6.1;I agree that the Gmail implementation is inconvenient, but I do not think that they are not compliant with RFC 4871 6.1 given the RFC 2119 definition of "SHOULD NOT". I should also mention that I''m not confident that stripping DKIM headers will resolve the problem. In fact, Gmail markes messages sent from ebay.com and paypal.com that do not pass DKIM validation as phishing [1][2][3]. I do not know if messages from amazon.com are handled similarly.> (b) Get your correspondents to use a non-broken email host;Lars, George - is that an option?> (c) Get the DKIM the spec changed or clarified;I think that RFC 4871 is pretty clear in the intent, but leaves room for interpretation via SHOULD / SHOULD NOT.> (d) Stop putting these abused things in your email headers.Obviously this isn''t going to happen. The amazon.com domain is a popular target for spammers and phishers, and providing DKIM headers may help prevent phishing attacks.> That would be better than asking lists.xen.org to start violating the > specified protocol. Now of course a SHOULD is not an absolute > requirement. Perhaps mailing lists are a special case somehow; but if > so I would expect this to be addressed in the relevant standards > documents. I don''t see any particular reason to think that > lists.xen.org is somehow unusual.Ultimately I think that Mailman should verify DKIM signatures, provide a new signature for the modified message (or have the outbound MTA do the signing), and retain the origional DKIM signature as a trace. I believe that this is in line with the recomendations for intermediary email handlers like Mailman in RFC 5863 [4]. Of course, I don''t know if Gmail will rework their implementation to ignore the invalid signature. At least one Mailman user reported success simply adding a new signature and not stripping any header [5]. If a test of removing DKIM headers to see if it helps with delivery to Gmail is off the table, then perhaps configuring Mailman in a way that doesn''t break DKIM signatures would be an option? Amazon''s signed headers include date, from, to, cc, subject, message-id and mime-version. If the subject manipulation of adding [Xen-devel] was removed, the signature would likely still be valid. Personally, I think that stripping DKIM headers as a short term workaround is less objectionable. Matt [1] http://gmailblog.blogspot.com/2008/07/fighting-phishing-with-ebay-and-paypal.html [2] https://support.google.com/mail/bin/answer.py?hl=en&answer=105760 [3] https://support.google.com/mail/bin/answer.py?hl=en&answer=175365 [4] http://tools.ietf.org/html/rfc5863#page-25 [5] http://mail.python.org/pipermail/mailman-users/2011-October/072304.html
Matt Wilson writes ("Re: [Xen-devel] lists.xen.org Mailman configuration
and DKIM"):> On Fri, Aug 03, 2012 at 07:44:30AM -0700, Ian Jackson wrote:
> > That would be better than asking lists.xen.org to start violating the
> > specified protocol. Now of course a SHOULD is not an absolute
> > requirement. Perhaps mailing lists are a special case somehow; but if
> > so I would expect this to be addressed in the relevant standards
> > documents. I don''t see any particular reason to think that
> > lists.xen.org is somehow unusual.
>
> Ultimately I think that Mailman should verify DKIM signatures, provide
> a new signature for the modified message (or have the outbound MTA do
> the signing), and retain the origional DKIM signature as a trace. I
> believe that this is in line with the recomendations for intermediary
> email handlers like Mailman in RFC 5863 [4]. Of course, I don''t
know
> if Gmail will rework their implementation to ignore the invalid
> signature. At least one Mailman user reported success simply adding a
> new signature and not stripping any header [5].
The solution to the broken DKIM implementations, or broken spec, must
not be allowed to become "install more DKIM". That is making the
problem worse, not better.
> Personally, I think that stripping DKIM headers as a short term
> workaround is less objectionable.
So bottom line is you think that Gmail is violating a SHOULD NOT.
And you are suggesting that the right fix for this is for us to also
violate a SHOULD NOT. That can''t be right.
> If a test of removing DKIM headers to see if it helps with delivery to
> Gmail is off the table, then perhaps configuring Mailman in a way that
> doesn''t break DKIM signatures would be an option?
Amazon''s signed
> headers include date, from, to, cc, subject, message-id and
> mime-version. If the subject manipulation of adding [Xen-devel] was
> removed, the signature would likely still be valid.
I don''t think that would be popular and I don''t think this is
a good
reason to do it.
Personally I think these subject line prefixes are annoying and if it
were my list it wouldn''t have had them to start with. But if you want
us to turn that off I think you need to get consensus for that.
Ian.
On 03/08/12 21:51, Matt Wilson wrote:>> (b) Get your correspondents to use a non-broken email host; > Lars, George - is that an option?Gmail is the best interface I''ve seen so far for dealing with a list like xen-devel. Giving it up would be a major downgrade. I''ve instructed gmail to white-list your e-mail address, so I (hopefully) shouldn''t be missing any more of your e-mails (although I may miss some from your colleagues). -George
On Mon, Aug 06, 2012 at 07:31:32AM -0700, Ian Jackson wrote:> Matt Wilson writes ("Re: [Xen-devel] lists.xen.org Mailman configuration and DKIM"): > > On Fri, Aug 03, 2012 at 07:44:30AM -0700, Ian Jackson wrote: > > > That would be better than asking lists.xen.org to start violating the > > > specified protocol. Now of course a SHOULD is not an absolute > > > requirement. Perhaps mailing lists are a special case somehow; but if > > > so I would expect this to be addressed in the relevant standards > > > documents. I don''t see any particular reason to think that > > > lists.xen.org is somehow unusual. > > > > Ultimately I think that Mailman should verify DKIM signatures, provide > > a new signature for the modified message (or have the outbound MTA do > > the signing), and retain the origional DKIM signature as a trace. I > > believe that this is in line with the recomendations for intermediary > > email handlers like Mailman in RFC 5863 [4]. Of course, I don''t know > > if Gmail will rework their implementation to ignore the invalid > > signature. At least one Mailman user reported success simply adding a > > new signature and not stripping any header [5]. > > The solution to the broken DKIM implementations, or broken spec, must > not be allowed to become "install more DKIM". That is making the > problem worse, not better.That''s possibly true, but I''m not well versed enough in the DKIM and related specs to say if "install more DKIM" makes things worse or better.> > Personally, I think that stripping DKIM headers as a short term > > workaround is less objectionable. > > So bottom line is you think that Gmail is violating a SHOULD NOT. > And you are suggesting that the right fix for this is for us to also > violate a SHOULD NOT. That can''t be right.Andrew Cooper helpfully pointed out that the actual problem is a DMARC policy advertised by amazon.com that requests all messages pass DKIM checks: $ dig +short TXT _dmarc.amazon.com. "v=DMARC1\; p=quarantine\; pct=100\; rua=mailto:dmarc-reports@bounces.amazon.com\; ruf=mailto:dmarc-reports@bounces.amazon.com" Gmail will treat a message with no DKIM signature from amazon.com the same as a broken DKIM signature from amazon.com, so stripping the headers won''t actually help here.> > If a test of removing DKIM headers to see if it helps with delivery to > > Gmail is off the table, then perhaps configuring Mailman in a way that > > doesn''t break DKIM signatures would be an option? Amazon''s signed > > headers include date, from, to, cc, subject, message-id and > > mime-version. If the subject manipulation of adding [Xen-devel] was > > removed, the signature would likely still be valid. > > I don''t think that would be popular and I don''t think this is a good > reason to do it. > > Personally I think these subject line prefixes are annoying and if it > were my list it wouldn''t have had them to start with. But if you want > us to turn that off I think you need to get consensus for that.The DMARC FAQ, http://dmarc.org/faq.html, has only this advice to mailing list operators: I operate a mailing list, what should I do? DMARC introduces the concept of aligned identifiers. It means the domain in the from header must match the d= in the DKIM signature and the domain in the mail from envelope. You have a few solutions: * operate as a strict forwarder, where the message is not changed and the validity of the DKIM signature is preserved * introduce an "Original Authentication Results" header to indicate you have performed the authentication and you are validating it * take ownership of the email, by removing the DKIM signature and putting your own as well as changing the from header in the email to contain an email address within your mailing list domain. None of these options are terribly compelling to me. I think that Mailman could run as a strict forwarder if the subject tags and message footers were disabled, but you''re right that we''d need to get consensus to make that change. The "Original Authentication Results" option sounds like yet another header that will have non-standard handling depending the implementer. Since modifying the Xen.org mailman configurations would only address the problem on one list server, I''ll investigate alternative solutions on the Amazon side of things. It seems that Google has a googlers.com domain for this type of thing [1]. Thanks again, Matt [1] http://mail.python.org/pipermail/mailman-developers/2011-December/021640.html