dovecot - Jun 2018 - Header DATE field not returned when placed after MIME section

Hello,

I am using Dovecot 2.2.34p0 on OpenBSD 6.3 stable.

I am writting an application using the vmime (http://www.vmime.org)
library (x-ref issue: https://github.com/kisli/vmime/issues/199).

I noticed through testing over my own e-mail server that in some
situations the DATE field is not returned upon ENVELOPE or
HEADER.FIELDS request.

The e-mails at fault happen to have the DATE field at the very end of
the header, after the MIME section.

Example 1:

Delivered-To: 
Received: 
Return-Path:
Delivered-To: 
DKIM-Signature: 
From: 
To: 
Message-ID:
Subject:
MIME-Version: 1.0
Content-Type: text/html;charset=UTF-8
Content-Transfer-Encoding: 7bit
Date: Fri, 01 Jun 2018 17:44:44 -0400

Example 2:

Return-Path:
Delivered-To: 
Received: 
Delivered-To:
DKIM-Signature:
From: 
Reply-To: 
To: 
Subject: 
MIME-Version: 1.0
Content-Type: multipart/mixed; 
        boundary="----=_Part_338_1573405959.1527768044781"
X-Mailer: EMail_2.0
Date: Fri, 01 Jun 2018 17:36:01 -0400

Thanks in advance for the insight,

On 2 Jun 2018, at 1.03, Xavier Guerin <lists at applepine.org>
wrote:
> 
> Hello,
> 
> I am using Dovecot 2.2.34p0 on OpenBSD 6.3 stable.
> 
> I am writting an application using the vmime (http://www.vmime.org)
> library (x-ref issue: https://github.com/kisli/vmime/issues/199).
> 
> I noticed through testing over my own e-mail server that in some
> situations the DATE field is not returned upon ENVELOPE or
> HEADER.FIELDS request.
> 
> The e-mails at fault happen to have the DATE field at the very end of
> the header, after the MIME section.
Can you easily reproduce this with doveadm? I can't, tested with current
master and v2.2.34. :

% cat << EOF | doveadm save -u tss
Delivered-To:
Received:
Return-Path:
Delivered-To:
DKIM-Signature:
From:
To:
Message-ID:
Subject:
MIME-Version: 1.0
Content-Type: text/html;charset=UTF-8
Content-Transfer-Encoding: 7bit
Date: Fri, 01 Jun 2018 17:44:44 -0400

body
EOF

% doveadm fetch -u tss hdr.date mailbox inbox '*'
hdr.date: Fri, 01 Jun 2018 17:44:44 -0400

% doveadm fetch -u tss imap.envelope mailbox inbox '*'
imap.envelope: "Fri, 01 Jun 2018 17:44:44 -0400" "" NIL NIL
NIL NIL NIL NIL NIL ""

I tested also with your example 2 mail, but it also worked. Maybe there's
something else special in those mails. If you can reproduce it with the above
commands, send me the exact mail as attachment that you used?

> On Jun 6, 2018, at 1:41 PM, Timo Sirainen <tss at iki.fi> wrote:
> 
> On 2 Jun 2018, at 1.03, Xavier Guerin <lists at applepine.org> wrote:
>> 
>> Hello,
>> 
>> I am using Dovecot 2.2.34p0 on OpenBSD 6.3 stable.
>> 
>> I am writting an application using the vmime (http://www.vmime.org)
>> library (x-ref issue: https://github.com/kisli/vmime/issues/199).
>> 
>> I noticed through testing over my own e-mail server that in some
>> situations the DATE field is not returned upon ENVELOPE or
>> HEADER.FIELDS request.
>> 
>> The e-mails at fault happen to have the DATE field at the very end of
>> the header, after the MIME section.
> 
> Can you easily reproduce this with doveadm? I can't, tested with
current master and v2.2.34. :
> 
> % cat << EOF | doveadm save -u tss
> Delivered-To:
> Received:
> Return-Path:
> Delivered-To:
> DKIM-Signature:
> From:
> To:
> Message-ID:
> Subject:
> MIME-Version: 1.0
> Content-Type: text/html;charset=UTF-8
> Content-Transfer-Encoding: 7bit
> Date: Fri, 01 Jun 2018 17:44:44 -0400
> 
> body
> EOF
> 
> % doveadm fetch -u tss hdr.date mailbox inbox '*'
> hdr.date: Fri, 01 Jun 2018 17:44:44 -0400
> 
> % doveadm fetch -u tss imap.envelope mailbox inbox '*'
> imap.envelope: "Fri, 01 Jun 2018 17:44:44 -0400" "" NIL
NIL NIL NIL NIL NIL NIL ""
> 
> I tested also with your example 2 mail, but it also worked. Maybe
there's something else special in those mails. If you can reproduce it with
the above commands, send me the exact mail as attachment that you used?
I reproduced your experiment and it does work. I double checked the raw content
of the message and it turns out that the Date field has actually been omitted.

I originally used ?Evolution? to check the message source and apparently this
tool shows the Date field even if it is missing.

I believe this situation will have to be handled by the library I am using.

Thanks !