dovecot - Sep 2007 - SORT(DATE) and missing Date headers

Hello,

Occasionally, I get mail that's missing a Date header. The usual 
suspects are iTunes weekly mailings and NYTimes email-to-a-friend 
articles. I use qmail, which doesn't "fix" these malformed emails
by
adding a Date header (like Sendmail does), so when they get to my 
mailbox, they're just as badly formed as they were when they were 
sent, which is to say, they have no Date header. Unfortunately, this 
means that SORT (DATE) puts them at the beginning of the list---and in 
a mailbox with lots of old mail, that's usually the wrong spot.

Now, my email clients will do some basic guesswork about when the 
message was sent, usually by falling back on the timestamp of the 
first-added Received header. So, they will display a date. But they 
have to then sort all messages themselves in order to get a valid 
sorting. Would it be possible to get Dovecot to fall back to Received 
header parsing whenever a message is missing a Date header? The idea 
is that SORT (DATE) would then become useful even in the face of 
commonly malformed email. How hard would this be to hack into the 
current Dovecot source? Does this maybe exist already (e.g. as SORT 
(X-DATE)?

~Kyle
-- 
There are four boxes to be used in defense of liberty: soap, ballot, 
jury, and ammo. Please use in that order.
                                                      -- Ed Howdershelt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20070924/4696254a/attachment-0002.bin>

We recently encountered this with a new VOIP voicemail system.  However, 
using Thunderbird at least, the time the message file was written is 
used (probably using "INTERNALDATE").

This might only apply if you're using Maildir's, but it shows that the 
behavior you're seeing might be specific to Mutt.

Rich

Kyle Wheeler wrote:
> Hello,
> 
> Occasionally, I get mail that's missing a Date header. The usual 
> suspects are iTunes weekly mailings and NYTimes email-to-a-friend 
> articles. I use qmail, which doesn't "fix" these malformed
emails by
> adding a Date header (like Sendmail does), so when they get to my 
> mailbox, they're just as badly formed as they were when they were sent,
> which is to say, they have no Date header. Unfortunately, this means 
> that SORT (DATE) puts them at the beginning of the list---and in a 
> mailbox with lots of old mail, that's usually the wrong spot.
> 
> Now, my email clients will do some basic guesswork about when the 
> message was sent, usually by falling back on the timestamp of the 
> first-added Received header. So, they will display a date. But they have 
> to then sort all messages themselves in order to get a valid sorting. 
> Would it be possible to get Dovecot to fall back to Received header 
> parsing whenever a message is missing a Date header? The idea is that 
> SORT (DATE) would then become useful even in the face of commonly 
> malformed email. How hard would this be to hack into the current Dovecot 
> source? Does this maybe exist already (e.g. as SORT (X-DATE)?
> 
> ~Kyle

On Mon, 2007-09-24 at 14:38 -0500, Kyle Wheeler wrote:
> Would it be possible to get Dovecot to fall back to Received 
> header parsing whenever a message is missing a Date header? The idea 
> is that SORT (DATE) would then become useful even in the face of 
> commonly malformed email.
Unfortunately that would violate the SORT spec.
> How hard would this be to hack into the current Dovecot source?
Replace mail_get_date() calls in src/imap/imap-sort.c with something
like:

t = mail_get_date(..);
if (t == (time_t)-1) t = mail_get_received_date(..);

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20070925/59b8cb19/attachment-0002.bin>