dovecot - Sep 2020 - Bug: Dovecot appending "MISSING_DOMAIN" to fetch envelope responses

Hi all,

I'm seeing Dovecot include the string "MISSING_DOMAIN" in fetch 
envelope requests when an mailbox's `addr-spec` part does not have a 
`domain` part.

For example:
>  C: a022 uid fetch 40 (envelope rfc822.header)
>  S: * 5 FETCH (UID 40 ENVELOPE ("Sat, 15 Aug 2020 12:53:05 +1000"
> "test {{name}}" (("Michael Gratton" NIL
"mike" "vee.net")) (("Michael
> Gratton" NIL "mike" "vee.net")) (("Michael
Gratton" NIL "mike"
> "vee.net")) (("postgres" NIL "postgres"
"MISSING_DOMAIN")) NIL NIL
> NIL NIL) RFC822.HEADER {literal/244b})
I understand that RFC 5322 require a mailbox to have a domain part, 
however it is still possible for mail to contain mailboxes that do not. 
In these cases, Dovecot returning random string such as the above is 
harmful, since it not possible for IMAP clients to determine whether 
host name string is actually present or something that the IMAP server 
has made up.

As a result, an IMAP client has two options: Either display the string 
as-is, preventing it from providing an appropriately localised 
explanation or visual indication of the problem in the user interface, 
or simply avoid ever requesting IMAP envelopes from the server and 
always download and parse the complete RFC822 headers itself (as I 
believe Thunderbird does).

The former is untenable since "MISSING_DOMAIN" does not mean anything 
to non-English speakers, the latter is untenable since it unnecessarily 
inflates the of volume of data that must be transmitted over the wire 
to work around a server bug.

In these cases, or when any other part of a required IMAP address 
structure is missing, Dovecot should return the empty string, e.g. "".

Cheers,
//Mike

Dovecot details:
> *REDACTED*:~$ sudo dovecot -n
> # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.4 ()
> # OS: Linux 4.19.0-9-amd64 x86_64 Debian 10.4 ext4
> # Hostname: *REDACTED*
> first_valid_uid = *REDACTED*
> hostname = *REDACTED*
> last_valid_uid = *REDACTED*
> lda_original_recipient_header = X-Original-To
> lmtp_rcpt_check_quota = yes
> lmtp_save_to_detail_mailbox = yes
> mail_gid = *REDACTED*
> mail_home = *REDACTED*
> mail_location = mdbox:*REDACTED*:UTF-8
> mail_privileged_group = mail
> mail_server_admin = mailto:*REDACTED*
> mail_temp_dir = *REDACTED*
> mail_uid = *REDACTED*
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope 
> encoded-character vacation subaddress comparator-i;ascii-numeric 
> relational regex imap4flags copy include variables body enotify 
> environment mailbox date index ihave duplicate mime foreverypart 
> extracttext imapsieve vnd.dovecot.imapsieve
> namespace inbox {
>   inbox = yes
>   location >   mailbox Drafts {
>     auto = subscribe
>     special_use = \Drafts
>   }
>   mailbox Junk {
>     auto = subscribe
>     autoexpunge = 30 days
>     special_use = \Junk
>   }
>   mailbox Sent {
>     auto = subscribe
>     special_use = \Sent
>   }
>   mailbox Trash {
>     auto = subscribe
>     autoexpunge = 30 days
>     special_use = \Trash
>   }
>   prefix >   separator = .
>   type = private
> }
> passdb {
>   args = *REDACTED*
>   driver = ldap
> }
> plugin {
>   imapsieve_mailbox1_before = file:*REDACTED*
>   imapsieve_mailbox1_causes = COPY
>   imapsieve_mailbox1_name = Junk
>   imapsieve_mailbox2_before = file:*REDACTED*
>   imapsieve_mailbox2_causes = COPY
>   imapsieve_mailbox2_from = Junk
>   imapsieve_mailbox2_name = *
>   recipient_delimiter = +
>   sieve = file://*REDACTED*
>   sieve_before = /etc/dovecot/sieve-before.d
>   sieve_global_extensions = +vnd.dovecot.pipe 
> +vnd.dovecot.environment +vnd.dovecot.report
>   sieve_pipe_bin_dir = /etc/dovecot/sieve-scripts
>   sieve_plugins = sieve_imapsieve sieve_extprograms
>   sieve_quota_max_storage = 50M
> }
> postmaster_address = *REDACTED*
> protocols = " imap lmtp sieve"
> service auth-worker {
>   user = mail
> }
> service auth {
>   unix_listener *REDACTED* {
>     group = postfix
>     mode = 0666
>     user = postfix
>   }
> }
> service lmtp {
>   process_min_avail = 2
>   unix_listener *REDACTED* {
>     group = postfix
>     mode = 0600
>     user = postfix
>   }
>   user = mail
> }
> ssl = required
> ssl_cert = <*REDACTED*
> ssl_client_ca_dir = *REDACTED*
> ssl_dh = # hidden, use -P to show it
> ssl_key = # hidden, use -P to show it
> submission_host = *REDACTED*:587
> userdb {
>   args = uid=8
>   driver = static
> }
> protocol lmtp {
>   mail_plugins = " sieve"
> }
> protocol imap {
>   mail_plugins = " imap_sieve"
> }

-- 
Michael Gratton.
<https://mjog.vee.net>

Any word about this issue? Should I file a bug in an actual bug tracker 
or something?

//Mike

On Sat, 15 Aug, 2020 at 13:26, Michael Gratton <mike at vee.net>
wrote:
> Hi all,
> 
> I'm seeing Dovecot include the string "MISSING_DOMAIN" in
fetch
> envelope requests when an mailbox's `addr-spec` part does not have a 
> `domain` part.
> 
> For example:
> 
>>  C: a022 uid fetch 40 (envelope rfc822.header)
>>  S: * 5 FETCH (UID 40 ENVELOPE ("Sat, 15 Aug 2020 12:53:05
+1000"
>> "test {{name}}" (("Michael Gratton" NIL
"mike" "vee.net"))
>> (("Michael Gratton" NIL "mike"
"vee.net")) (("Michael Gratton" NIL
>> "mike" "vee.net")) (("postgres" NIL
"postgres" "MISSING_DOMAIN"))
>> NIL NIL NIL NIL) RFC822.HEADER {literal/244b})
> 
> I understand that RFC 5322 require a mailbox to have a domain part, 
> however it is still possible for mail to contain mailboxes that do 
> not. In these cases, Dovecot returning random string such as the 
> above is harmful, since it not possible for IMAP clients to determine 
> whether host name string is actually present or something that the 
> IMAP server has made up.
> 
> As a result, an IMAP client has two options: Either display the 
> string as-is, preventing it from providing an appropriately localised 
> explanation or visual indication of the problem in the user 
> interface, or simply avoid ever requesting IMAP envelopes from the 
> server and always download and parse the complete RFC822 headers 
> itself (as I believe Thunderbird does).
> 
> The former is untenable since "MISSING_DOMAIN" does not mean
anything
> to non-English speakers, the latter is untenable since it 
> unnecessarily inflates the of volume of data that must be transmitted 
> over the wire to work around a server bug.
> 
> In these cases, or when any other part of a required IMAP address 
> structure is missing, Dovecot should return the empty string, e.g.
"".
> 
> Cheers,
> //Mike
> 
> Dovecot details:
> 
>> *REDACTED*:~$ sudo dovecot -n
>> # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf
>> # Pigeonhole version 0.5.4 ()
>> # OS: Linux 4.19.0-9-amd64 x86_64 Debian 10.4 ext4
>> # Hostname: *REDACTED*
>> first_valid_uid = *REDACTED*
>> hostname = *REDACTED*
>> last_valid_uid = *REDACTED*
>> lda_original_recipient_header = X-Original-To
>> lmtp_rcpt_check_quota = yes
>> lmtp_save_to_detail_mailbox = yes
>> mail_gid = *REDACTED*
>> mail_home = *REDACTED*
>> mail_location = mdbox:*REDACTED*:UTF-8
>> mail_privileged_group = mail
>> mail_server_admin = mailto:*REDACTED*
>> mail_temp_dir = *REDACTED*
>> mail_uid = *REDACTED*
>> managesieve_notify_capability = mailto
>> managesieve_sieve_capability = fileinto reject envelope 
>> encoded-character vacation subaddress comparator-i;ascii-numeric 
>> relational regex imap4flags copy include variables body enotify 
>> environment mailbox date index ihave duplicate mime foreverypart 
>> extracttext imapsieve vnd.dovecot.imapsieve
>> namespace inbox {
>>   inbox = yes
>>   location >>   mailbox Drafts {
>>     auto = subscribe
>>     special_use = \Drafts
>>   }
>>   mailbox Junk {
>>     auto = subscribe
>>     autoexpunge = 30 days
>>     special_use = \Junk
>>   }
>>   mailbox Sent {
>>     auto = subscribe
>>     special_use = \Sent
>>   }
>>   mailbox Trash {
>>     auto = subscribe
>>     autoexpunge = 30 days
>>     special_use = \Trash
>>   }
>>   prefix >>   separator = .
>>   type = private
>> }
>> passdb {
>>   args = *REDACTED*
>>   driver = ldap
>> }
>> plugin {
>>   imapsieve_mailbox1_before = file:*REDACTED*
>>   imapsieve_mailbox1_causes = COPY
>>   imapsieve_mailbox1_name = Junk
>>   imapsieve_mailbox2_before = file:*REDACTED*
>>   imapsieve_mailbox2_causes = COPY
>>   imapsieve_mailbox2_from = Junk
>>   imapsieve_mailbox2_name = *
>>   recipient_delimiter = +
>>   sieve = file://*REDACTED*
>>   sieve_before = /etc/dovecot/sieve-before.d
>>   sieve_global_extensions = +vnd.dovecot.pipe 
>> +vnd.dovecot.environment +vnd.dovecot.report
>>   sieve_pipe_bin_dir = /etc/dovecot/sieve-scripts
>>   sieve_plugins = sieve_imapsieve sieve_extprograms
>>   sieve_quota_max_storage = 50M
>> }
>> postmaster_address = *REDACTED*
>> protocols = " imap lmtp sieve"
>> service auth-worker {
>>   user = mail
>> }
>> service auth {
>>   unix_listener *REDACTED* {
>>     group = postfix
>>     mode = 0666
>>     user = postfix
>>   }
>> }
>> service lmtp {
>>   process_min_avail = 2
>>   unix_listener *REDACTED* {
>>     group = postfix
>>     mode = 0600
>>     user = postfix
>>   }
>>   user = mail
>> }
>> ssl = required
>> ssl_cert = <*REDACTED*
>> ssl_client_ca_dir = *REDACTED*
>> ssl_dh = # hidden, use -P to show it
>> ssl_key = # hidden, use -P to show it
>> submission_host = *REDACTED*:587
>> userdb {
>>   args = uid=8
>>   driver = static
>> }
>> protocol lmtp {
>>   mail_plugins = " sieve"
>> }
>> protocol imap {
>>   mail_plugins = " imap_sieve"
>> }
> 
> 
> --
> Michael Gratton.
> <https://mjog.vee.net>

-- 
Michael Gratton.
<https://mjog.vee.net>

On 31 Aug 2020, at 06:23, Michael Gratton <mike at vee.net>
wrote:
> Any word about this issue? Should I file a bug in an actual bug tracker or
something?
I don't think it's a bug?



-- 
Eliot: We'll figure it out. We always do.
Margo: When it's a test to cheat on. Not when we're stuck in some epic
fantasy
       that likes to behead heroes halfway through season one. If we even are
       heroes. We might be comic relief.