Hello! I apologize for bringing perhaps trivial/well-known matter, but I am interested in your opinion. RFC 5322 clearly states that mail messages SHOULD contain a Message ID identifier, but if the do contain it, it MUST be globally unique. Despite this requirement, I have encountered senders (namely Spamcop) that sends obviously different (albeit related) messages called "Alert" and "Summary" (they are always related to the same incident and have the same Message ID). This creates all sorts of problems when processing these mails, namely with users that have local forwards from one domain to another (our mailserver hosts multiple domains), because for example Dovecot refuses to forward the second message, flagging it as a duplicate. My question to you is - did you also encounter similar incorrect (according to RFC standards) problem, and if so, is there a way to persuade dovecot to perhaps determine the uniqueness of a message by other means than just checking the message ID (i.e. look at other identifiers, Subject, perhaps)? Because according to the log records, Spamcop does not seem to be the only offender. Thanks in advance for any reactions, and if I did something wrong by writing this message, I apologize again in advance. If required, I can provide samples of the Spamcop messages. -- --===============-- --== Best Regards! ==-- --===============-- Daniel Ry?link Sysadmin @ Quantcom.cz Czech Republic
> I apologize for bringing perhaps trivial/well-known matter, but I am > interested in your opinion. > > RFC 5322 clearly states that mail messages SHOULD contain a Message ID > identifier, but if the do contain it, it MUST be globally unique. > > Despite this requirement, I have encountered senders (namely Spamcop) > that sends obviously different (albeit related) messages called "Alert" > and "Summary" (they are always related to the same incident and have the > same Message ID). This creates all sorts of problems when processing > these mails, namely with users that have local forwards from one domain > to another (our mailserver hosts multiple domains), because for example > Dovecot refuses to forward the second message, flagging it as a duplicate. > > My question to you is - did you also encounter similar incorrect > (according to RFC standards) problem, and if so, is there a way to > persuade dovecot to perhaps determine the uniqueness of a message by > other means than just checking the message ID (i.e. look at other > identifiers, Subject, perhaps)? Because according to the log records, > Spamcop does not seem to be the only offender. >I would think this is more related to MTA's then dovecot. It is dovecot's core job to put messages in mailboxes. However interesting this globally unique. At first sight, I would say a bit unnecessarily broad. I would recommend using a mail filter in any mta, grab whatever you want and analyze that, with that solution you can add any header you like. I do not get what spamcop has to do with this, afaik this is a dnsbl.
On 27.01.22 16:17, Daniel Ry?link wrote:> RFC 5322 clearly states that mail messages SHOULD contain a Message ID > identifier, but if the do contain it, it MUST be globally unique.The problem with that requirement being that it remains unclear how long the mail( copie)s it's attached to remain interchangeable versions of a "globally unique" message. When an e-mail sent to a at b.com and c at d.net gets split into two copies and the separate mailservers for b.com and d.net each forward a copy *as is* to e at f.net, the f.net server would be entirely correct to call whichever arrives second a duplicate, even though they'll differ by at least the Received: headers. When c at d.net is a (simplistic) mailinglist, however, AFAICT it is still considered proper that the copy it sends to e at f.net retains the original Message-ID - even though there will be more extensive changes to the headers (list headers, Reply-To:, possibly stuff like retrofitting SPF and DKIM, ...). Assuming that the list mail arrives second at f.net, deduplication will keep the recipient from ever reaping the benefit of those changed headers (as in, having a "Reply List" button pop up in TB). (However, if I understand correctly that on the list you're talking about you see the same Message-ID applied to e-mails that are essentially *replies*/followups to the original one with entirely different content, I suppose that most people will agree that they *should* each have a Message-ID of their own, with the IDs of the earlier e-mails appearing in In-Reply-To: and References: headers to support threading in MUAs.) Kind regards, -- Jochen Bern Systemingenieur Binect GmbH -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3449 bytes Desc: S/MIME Cryptographic Signature URL: <https://dovecot.org/pipermail/dovecot/attachments/20220128/14ea71ca/attachment.bin>
On January 27, 2022 6:17:05 AM AKST, "Daniel Ry?link" <daniel.ryslink at quantcom.cz> wrote:> >RFC 5322 clearly states that mail messages SHOULD contain a Message ID >identifier, but if the do contain it, it MUST be globally unique. >That's nice polite behavior, all right, but the enforcement of it is another matter entirely. Slap a tracking label with a barcode on a piece of mail, and the mail truck is taking off from the loading dock at the post office with the door wide open being rear-ended by the cop car with a federal warrant and and a razor-sharp military letter opener in his hand. Oh yeah, I almost forgot I've got a flat tire and I discovered my brake hose was apparently slit wide open, and my att0rney says I'm facing additional charges since they had a lawful warrant to take all that action against me on my account. /sarcasm>Despite this requirement, I have encountered senders (namely Spamcop) >that sends obviously different (albeit related) messages called "Alert" >and "Summary" (they are always related to the same incident and have the >same Message ID). This creates all sorts of problems when processing >these mails, namely with users that have local forwards from one domain >to another (our mailserver hosts multiple domains), because for example >Dovecot refuses to forward the second message, flagging it as a duplicate. > >My question to you is - did you also encounter similar incorrect >(according to RFC standards) problem, and if so, is there a way to >persuade dovecot to perhaps determine the uniqueness of a message by >other means than just checking the message ID (i.e. look at other >identifiers, Subject, perhaps)? Because according to the log records, >Spamcop does not seem to be the only offender. >Thank you, that's a years-old bug, pet peeve and aggravation in several mailing systems not just Dovecot and you get my upvote for the question and complaint. We need to be nice, and deal respectfully but set our limits with people who aren't being so nice when they send emails.>Thanks in advance for any reactions, and if I did something wrong by >writing this message, I apologize again in advance. > >If required, I can provide samples of the Spamcop messages. >I am hoping there are more and better solutions to this problem forthcoming. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.