SMTPUTF8 support

André Rodier
2019-04-05T19:47:33Z

On 2019-04-05 08:57, David B?rgin via dovecot wrote:
> Andr?, are you quite sure you have it working?
>
> In this thread someone from Open-Xchange stated that no, Dovecot
> doesn?t
> have SMTPUTF8 support implemented, and the same response was given by
> another Dovecot developer last September (it ?is being considered? was
> the answer then, see
> https://dovecot.org/pipermail/dovecot/2018-September/112887.html).
>
> I am using LMTP to deliver mail to Dovecot from Postfix, and delivery
> fails with Postfix reporting: ?SMTPUTF8 is required, but was not
> offered
> by host mail.my.org[private/dovecot-lmtp]?. I doubt that this can work
> without a change in Dovecot LMTP.
>
> Anyway, I am also interested in SMTPUTF8 support, so +1 from me!

Hello David,

Well, it seems to work from one Homebox to another, for instance:

Here the server logs:
----------------------------------
> Apr 05 19:20:46 portal postfix/qmgr[21470]: 6AC3240807:
> from=<andre at rodier.me>, size=3066, nrcpt=1 (queue active)
> Apr 05 19:20:46 portal clamsmtpd[14542]: 10004D: from=andre at rodier.me,
> to=andr?@homebox.space, status=CLEAN
> Apr 05 19:20:46 portal postfix/smtp[30761]: 1FCA540467:
> to=<andr?@homebox.space>, relay=127.0.0.1[127.0.0.1]:34001,
delay=1.4,
> delays=0.73/0.02/0.07/0.59, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued
> as 6AC3240807)
> Apr 05 19:20:46 portal postfix/smtpd[30763]: disconnect from
> localhost[127.0.0.1] ehlo=1 xforward=2 mail=1 rcpt=1 data=1 quit=1
> commands=7
> Apr 05 19:20:46 portal postfix/qmgr[21470]: 1FCA540467: removed
> Apr 05 19:20:46 portal dovecot[12813]: lmtp(30766): Connect from local
> Apr 05 19:20:46 portal dovecot[12813]: lmtp(andre):
> WGTdAo6qp1wueAAAeNB+QA: sieve:
> msgid=<7d53-5ca7aa80-1-51005a80 at 31476855>: stored mail into
mailbox
> 'INBOX'
> Apr 05 19:20:46 portal postfix/lmtp[30765]: 6AC3240807:
> to=<andre at homebox.space>, orig_to=<andr?@homebox.space>,
> relay=portal.homebox.space[private/dovecot-lmtp], delay=0.62,
> delays=0.59/0.01/0.01/0.01, dsn=2.0.0, status=sent (250 2.0.0
> <andre at homebox.space> WGTdAo6qp1wueAAAeNB+QA Saved)
> Apr 05 19:20:46 portal dovecot[12813]: lmtp(30766): Disconnect from
> local: Successful quit
> Apr 05 19:20:46 portal postfix/qmgr[21470]: 6AC3240807: removed
----------------------------------

And the message source:
----------------------------------
> Return-Path: <andre at rodier.me>
> Delivered-To: andre at homebox.space
> Received: from portal.homebox.space
> 	by portal (Dovecot) with LMTP id WGTdAo6qp1wueAAAeNB+QA
> 	for <andre at homebox.space>; Fri, 05 Apr 2019 19:20:46 +0000
> Received: from portal.homebox.space (localhost [127.0.0.1])
> 	by portal.homebox.space (Postfix) with ESMTP id 6AC3240807
> 	for <andr??@homebox.space>; Fri,  5 Apr 2019 19:20:45 +0000 (UTC)
> Authentication-Results: portal.homebox.space;
> 	dkim=pass (4096-bit key; unprotected) header.d=rodier.me
> header.i=@rodier.me header.b="bNaIhJcQ";
> 	dkim-atps=neutral
> Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=92.19.253.42;
> helo=osaka.rodier.me; envelope-from=andre at rodier.me;
receiver=<UNKNOWN>
> Received: from osaka.rodier.me (host-92-19-253-42.static.as13285.net
> [92.19.253.42])
> 	by portal.homebox.space (Postfix) with ESMTPS id 1FCA540467
> 	for <andr??@homebox.space>; Fri,  5 Apr 2019 19:20:44 +0000 (UTC)
> Received: from osaka.rodier.me (localhost [127.0.0.1])
> 	by osaka.rodier.me (Postfix) with ESMTP id 68658201DE
> 	for <andr??@homebox.space>; Fri,  5 Apr 2019 20:20:43 +0100 (BST)
> Received: from localhost (localhost [127.0.0.1])
> 	by osaka.rodier.me (Postfix) with ESMTPA id 32C1A200CC
> 	for <andr??@homebox.space>; Fri,  5 Apr 2019 20:20:43 +0100 (BST)
> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rodier.me;
> s=main;
> 	t=1554492043; bh=ojnzW/eF3u3/ihaSAhQfvUWEX7Blx4fCTTVCC9mfgZE=;
> 	h=From:Date:To:Subject:From;
> 	b=bNaIhJcQtD7/IbjI/VGUIWpOk1QyYioKsgsCva8sL040L7VDMmLgfbQ5TWFwO/wub
> 	 +8THBgOTWj/39FyoKQJYk0BahQ7Z4LkUrH20Jcq5yDEOC7q/xfwIwUV/sUaj6C3wLZ
> 	 VGUp8Fg2oVDD+ciGEnfa2OuEC/LajvmXoDqUHHtSwTFZvfkh1nRDqxcNd/FSyvYQe8
> 	 frgBySFgzI+xTWVG7Oa07CQPG+HenqwrPyaZD0SNpHnrvH5ICdAriv2UMKgQnsWO80
> 	 aSz4EDySdwMah17kfh4u06GlXd3duQCN8hyxBBDjcI879N/+CQM3Uz1NTbIDe82Lf7
> 	 bSUJzvMczCdZFm5J84hvLd+j85pY7T9052JTcUANUQPnTc1Q3yrQFC3QuKD8Amb8VO
> 	 qHR0q0OP6SYqoQ0g/moMF2FhWaQg3pCRpIEOS73Ncnnf+fJGQJnPi8G8YcLVLZCDCx
> 	 RlSauA/YuF5tiIBHFLdjtlgVgwk38tT8ck+D8kS3BAB4nXBVlreCiK0vMK8HyoCpsC
> 	 LMJ0e5KhBzCDXOPvSLiohn2zcmYpa9DywmWFVTZNx3INzTH7nsa40AYGFl+k+G+pe+
> 	 ej+MRDwmdOePfl3DGGEgpn0JMNiAmECe6gHA+/N1TTk5SbbiKLNh8KXzrZECWVrvUx
> 	 55kqY/LPuQEwxCnA/VZsQbDk> Content-Type: multipart/alternative;
>
boundary="----=_=-_OpenGroupware_org_NGMime-32083-1554492042.981625-0------"
>  From: =?utf-8?q?Andr=C3=A9_Rodier?= <andre at rodier.me>
> X-Forward: 127.0.0.1
> Date: Fri, 05 Apr 2019 20:20:42 +0100
> To: =?utf-8?q?andr=E9=40homebox=2Espace?=@rodier.me
> MIME-Version: 1.0
> Message-ID: <7d53-5ca7aa80-1-51005a80 at 31476855>
> Disposition-Notification-To: andre at rodier.me
> Subject: test 2020
> User-Agent: SOGoMail
> X-AV-Checked: ClamSMTP (Client IP: 127.0.0.1)
> Authentication-Results: portal.homebox.space;
> 	spf=temperror (portal.homebox.space: error in processing during lookup
> of andre at rodier.me: DNS error) smtp.mailfrom=andre at rodier.me
> X-Spam-Status: Yes, score=6.61
> X-AV-Checked: ClamSMTP (Client IP: 127.0.0.1)
> X-Spam: Yes
>
> ------=_=-_OpenGroupware_org_NGMime-32083-1554492042.981625-0------
> Content-Type: text/plain; charset=utf-8
> Content-Transfer-Encoding: 7bit
> Content-Length: 15
>
>
> This is a test
>
> ------=_=-_OpenGroupware_org_NGMime-32083-1554492042.981625-0------
> Content-Type: text/html; charset=utf-8
> Content-Transfer-Encoding: 7bit
> Content-Length: 27
>
> This is a test
> ------=_=-_OpenGroupware_org_NGMime-32083-1554492042.981625-0--------
----------------------------------

However, it seems that mails from GMails are not received, and I have an error
message from the postfix mailter:

> Apr  5 10:16:39 portal postfix/smtpd[2566]: Anonymous TLS connection
established from mail-lf1-f51.google.com[209.85.167.51]: TLSv1.2 with cipher
ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
> Apr  5 10:16:39 portal policyd-spf[2574]: prepend Received-SPF: Pass
(mailfrom) identity=mailfrom; client-ip=209.85.167.51;
helo=mail-lf1-f51.google.com; envelope-from=andre.rodier at gmail.com;
receiver=<UNKNOWN>
> Apr  5 10:16:39 portal postfix/smtpd[2566]: EB38240467:
client=mail-lf1-f51.google.com[209.85.167.51]
> Apr  5 10:16:40 portal postfix/cleanup[2576]: EB38240467:
message-id=<CAKr9_Nv1pgDAC31cFF4YPE-mnYxSusmKsWHOCxWPSeJ2Adj8cQ at
mail.gmail.com>
> Apr  5 10:16:40 portal opendkim[15477]: EB38240467: mail-lf1-f51.google.com
[209.85.167.51] not internal
> Apr  5 10:16:40 portal opendkim[15477]: EB38240467: not authenticated
> Apr  5 10:16:40 portal opendkim[15477]: EB38240467: DKIM verification
successful
> Apr  5 10:16:40 portal opendkim[15477]: EB38240467: s=20161025 d=gmail.com
SSL
> Apr  5 10:16:40 portal opendmarc[16548]: implicit authentication service:
portal.homebox.space
> Apr  5 10:16:40 portal opendmarc[16548]: EB38240467: gmail.com pass
> Apr  5 10:16:40 portal postfix/cleanup[2576]: EB38240467: milter-reject:
END-OF-MESSAGE from mail-lf1-f51.google.com[209.85.167.51]: 4.7.1 Try again
later; from=<andre.rodier at gmail.com> to=<andr?@homebox.space>
proto=ESMTP helo=<mail-lf1-f51.google.com>
> Apr  5 10:16:40 portal postfix/smtpd[2566]: disconnect from
mail-lf1-f51.google.com[209.85.167.51] ehlo=2 starttls=1 mail=1 rcpt=1 data=0/1
quit=1 commands=6/7

Thanks to you, I have another bug to chase with gmail and postfix
cleanup. I could ask Google to use HomeBox, but I am not sure they
would agree ;-)

Regarding your issue, there might be many causes.

I remember one, with the dovecot, for the Quota plugin that failed to lookup the
user, I had to add "quota_status_nouser = DUNNO",
so it does not fail when the user is not found (if I understood well).

Another note, I am using LDAP lookups, _and_ a specific schema for unicode email
addresses. Otherwise, this would not work:

root at portal:/etc/postfix# cat ldap-aliases.cf
# Basic LDAP settings for postfix
server_host = ldap://ldap.homebox.space/
start_tls = yes

search_base = dc=homebox,dc=space
search_scope = sub

# Allow advanced LDAP search with the recipient delimiter
query_filter = (|(mail=%s)(intlMailAddr=%s))
result_attribute = mail

# Force support for UTF8
version = 3

Thanks to this, ldap lookup is unicode compliant:

> root at portal:/etc/postfix# postmap -q andr?@homebox.space
ldap:/etc/postfix/ldap-aliases.cf
> andre at homebox.space
>

The LDAP schema for internationalised email addresses:

> # The attribute type 'intlMailAddr' is defined for storing SMTPUTF8
> # compliant addresses [RFC6530]
> # https://tools.ietf.org/html/draft-stroeder-mailboxrelatedobject-07
>
> dn: cn=intlMailAddr,cn=schema,cn=config
> objectClass: olcSchemaConfig
> cn: intlMailAddr
> olcAttributeTypes: ( 1.3.6.1.4.1.5427.1.389.4.18
>   NAME 'intlMailAddr'
>   DESC 'Internationalized Email Address'
>   EQUALITY caseIgnoreMatch
>   SUBSTR caseIgnoreSubstringsMatch
>   SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )
> olcObjectClasses: ( 1.3.6.1.4.1.5427.1.389.6.9
>   NAME 'mailboxRelatedObject'
>   DESC 'Associated RFC 5321 mailbox for any entry'
>   AUXILIARY
>   MAY ( displayName $ mail $ intlMailAddr ) )
>

As I wrote earlier, All the software in the chain should be UTF-8 ready

--
Andr? Rodier
HomeBox: https://github.com/progmaticltd/homebox