dovecot - Mar 2018 - [ext] Re: Panic: file smtp-address.c: line 530 (smtp_address_write): assertion failed: (smtp_char_is_qpair(*p))

I have an email which cannot be delivered using LMTP:

Mar  2 15:26:54 mail-cbf dovecot: lmtp(backup at
backup.invalid)<29736><cPd5Mi5fmVoodAAAplP5LA>: Panic: file
smtp-address.c: line 530 (smtp_address_write): assertion failed:
(smtp_char_is_qpair(*p))
Mar  2 15:26:54 mail-cbf dovecot: lmtp(backup at
backup.invalid)<29736><cPd5Mi5fmVoodAAAplP5LA>: Error: Raw
backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xc6aca) [0x7f7fb50d3aca] ->
/usr/lib/dovecot/libdovecot.so.0(+0xc6bad) [0x7f7fb50d3bad] ->
/usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f7fb5045721] ->
/usr/lib/dovecot/libdovecot.so.0(smtp_address_write+0x21d)
[0x7f7fb504931d] ->
/usr/lib/dovecot/libdovecot.so.0(smtp_address_encode+0x21)
[0x7f7fb5049411] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x3774)
[0x7f7fb56d0774] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x3abf)
[0x7f7fb56d0abf] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_finish+0x7c)
[0x7f7fb53c858c] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x104)
[0x7f7fb53baeb4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_copy+0x46)
[0x7f7fb53e13f6] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x3999)
[0x7f7fb56d0999] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x48956)
[0x7f7fb53c8956] ->
/usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver_save+0x1ac)
[0x7f7fb56d12ac] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x1f6)
[0x7f7fb56d1916] -> dovecot/lmtp(lmtp_local_data+0x610)
[0x560dfd665dc0] -> dovecot/lmtp(cmd_data_continue+0x233)
[0x560dfd664b53] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a6e0)
[0x7f7fb50576e0] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69)
[0x7f7fb50eb649] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109)
[0x7f7fb50ecf29] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52)
[0x7f7fb50eb752] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7f7fb50eb968] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f7fb50682a3] -> dovecot/lmtp(main+0x23d) [0x560dfd663c3d] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
[0x7f7fb4c63830] -> dovecot/lmtp(_start+0x29) [0x560dfd663d79]
Mar  2 15:26:55 mail-cbf dovecot: lmtp(backup at
backup.invalid)<29736><cPd5Mi5fmVoodAAAplP5LA>: Fatal: master:
service(lmtp): child 29736 killed with signal 6 (core dumped)

Using  dovecot 2:2.3.0.1-6 packages on Ubuntu 16.04

The address causing the error is:

From: =?utf-8?Q?Dorit_M=C3=BCller?= <d.m?ller at JOMEC.de>

Note the "umlaut" in the email address... :)
-- 
Ralf Hildebrandt
  Gesch?ftsbereich IT | Abteilung Netzwerk
  Charit? - Universit?tsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebrandt at charite.de | https://www.charite.de

Op 3/2/2018 om 3:32 PM schreef Ralf Hildebrandt:
> I have an email which cannot be delivered using LMTP:
>
> Mar  2 15:26:54 mail-cbf dovecot: lmtp(backup at
backup.invalid)<29736><cPd5Mi5fmVoodAAAplP5LA>: Panic: file
smtp-address.c: line 530 (smtp_address_write): assertion failed:
(smtp_char_is_qpair(*p))
> Mar  2 15:26:54 mail-cbf dovecot: lmtp(backup at
backup.invalid)<29736><cPd5Mi5fmVoodAAAplP5LA>: Error: Raw
backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xc6aca) [0x7f7fb50d3aca] ->
/usr/lib/dovecot/libdovecot.so.0(+0xc6bad) [0x7f7fb50d3bad] ->
> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f7fb5045721] ->
/usr/lib/dovecot/libdovecot.so.0(smtp_address_write+0x21d)
> [0x7f7fb504931d] ->
/usr/lib/dovecot/libdovecot.so.0(smtp_address_encode+0x21)
> [0x7f7fb5049411] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x3774)
> [0x7f7fb56d0774] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x3abf)
> [0x7f7fb56d0abf] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_finish+0x7c)
> [0x7f7fb53c858c] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x104)
> [0x7f7fb53baeb4] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mdbox_copy+0x46)
> [0x7f7fb53e13f6] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x3999)
> [0x7f7fb56d0999] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x48956)
> [0x7f7fb53c8956] ->
/usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver_save+0x1ac)
> [0x7f7fb56d12ac] ->
/usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x1f6)
> [0x7f7fb56d1916] -> dovecot/lmtp(lmtp_local_data+0x610)
> [0x560dfd665dc0] -> dovecot/lmtp(cmd_data_continue+0x233)
> [0x560dfd664b53] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a6e0)
> [0x7f7fb50576e0] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69)
> [0x7f7fb50eb649] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109)
> [0x7f7fb50ecf29] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52)
> [0x7f7fb50eb752] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
> [0x7f7fb50eb968] ->
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
> [0x7f7fb50682a3] -> dovecot/lmtp(main+0x23d) [0x560dfd663c3d] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
> [0x7f7fb4c63830] -> dovecot/lmtp(_start+0x29) [0x560dfd663d79]
> Mar  2 15:26:55 mail-cbf dovecot: lmtp(backup at
backup.invalid)<29736><cPd5Mi5fmVoodAAAplP5LA>: Fatal: master:
service(lmtp): child 29736 killed with signal 6 (core dumped)
>
> Using  dovecot 2:2.3.0.1-6 packages on Ubuntu 16.04
>
> The address causing the error is:
>
> From: =?utf-8?Q?Dorit_M=C3=BCller?= <d.m?ller at JOMEC.de>
>
> Note the "umlaut" in the email address... :)
We are looking into it.

Regards,

Stephan.

Hi Ralf,

Op 3/2/2018 om 3:32 PM schreef Ralf Hildebrandt:
> I have an email which cannot be delivered using LMTP:
>
> Mar  2 15:26:54 mail-cbf dovecot: lmtp(backup at
backup.invalid)<29736><cPd5Mi5fmVoodAAAplP5LA>: Panic: file
smtp-address.c: line 530 (smtp_address_write): assertion failed:
(smtp_char_is_qpair(*p))
> Mar  2 15:26:54 mail-cbf dovecot: lmtp(backup at
backup.invalid)<29736><cPd5Mi5fmVoodAAAplP5LA>: Error: Raw
backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xc6aca) [0x7f7fb50d3aca] ->
/usr/lib/dovecot/libdovecot.so.0(+0xc6bad)
A panic like that is always a bad thing and it should be fixed. Question
is how this should be addressed.
> Using  dovecot 2:2.3.0.1-6 packages on Ubuntu 16.04
>
> The address causing the error is:
>
> From: =?utf-8?Q?Dorit_M=C3=BCller?= <d.m?ller at JOMEC.de>
>
> Note the "umlaut" in the email address... :)
Clearly, the relevant specifications don't allow UTF-8 in the local part
without email address internationalization (EAI), which Dovecot does not
support nor announce yet (although that should be mended somewhat soon).
My preferred fix for now would be to reject addresses like that, which
would maybe still mean that this message is rejected entirely.
Alternatively, that address at least needs to be skipped, ignored, or
modified: all of which aren't very nice things to do. The alternative is
forwarding this violation to other systems, which is seldom acceptable
either.

But first a few questions:

1) Who or what is sending messages like that (without the proper
capability support available at the server side)?
2) Why would an MTA accept this message? What software is doing that
(and why)?
3) What does the envelope address look like? Do both header and envelope
have that umlaut (unlikely, because Dovecot should reject that already)?
Maybe this could be fixed by substituting the envelope address if it is
sufficiently similar.
4) Where does this fail actually? At some point it is trying to
construct an SMTP (RFC 5321) address from the header address (RFC 5322).
Do you have a GDB backtrace with proper symbols?

Regards,

Stephan.

> 1) Who or what is sending messages like that (without the proper
> capability support available at the server side)?
I think that was some sort of commercial mass mailer:

List-Unsubscribe:  <http://newsletter.jomec.de/rmftlp.php?cid=...>
> 2) Why would an MTA accept this message? What software is doing that
> (and why)?
Postfix. It was just in the headers, so it was not relevvant for delivery.
> 3) What does the envelope address look like? Do both header and envelope
> have that umlaut (unlikely, because Dovecot should reject that already)?
> Maybe this could be fixed by substituting the envelope address if it is
> sufficiently similar.
Envelope sender is: bounce+15396 at bounce.crsend.com
> 4) Where does this fail actually? At some point it is trying to
> construct an SMTP (RFC 5321) address from the header address (RFC 5322).
> Do you have a GDB backtrace with proper symbols?
Don't have a backtrace yet, but since the mail is still in the queue I
could try getting one.

-- 
Ralf Hildebrandt
  Gesch?ftsbereich IT | Abteilung Netzwerk
  Charit? - Universit?tsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebrandt at charite.de | https://www.charite.de

On 03/02/2018 03:32 PM, Ralf Hildebrandt wrote:
> The address causing the error is:
> 
> From: =?utf-8?Q?Dorit_M=C3=BCller?= <d.m?ller at JOMEC.de>
> 
> Note the "umlaut" in the email address... :)
This is about SMTPUTF8 (RFC6531). Looks like your only option is to disable
smtputf8_enable in Postfix config. Of course Dovecot should never panic, so a
fix for this
would be nice, separately from the SMTPUTF8 support.

-- 
Aleksander 'A.L.E.C' Machniak
Kolab Groupware Developer         [http://kolab.org]
Roundcube Webmail Developer   [http://roundcube.net]
----------------------------------------------------
PGP: 19359DC1 # Blog: https://kolabian.wordpress.com

On 04/03/18 08:08, A.L.E.C wrote:
> On 03/02/2018 03:32 PM, Ralf Hildebrandt wrote:
>> The address causing the error is:
>>
>> From: =?utf-8?Q?Dorit_M=C3=BCller?= <d.m?ller at JOMEC.de>
>>
>> Note the "umlaut" in the email address... :)
> This is about SMTPUTF8 (RFC6531). Looks like your only option is to disable
> smtputf8_enable in Postfix config. Of course Dovecot should never panic, so
a fix for this
> would be nice, separately from the SMTPUTF8 support.
>
Postfix already permitted UTF8 in message headers and local part of the
address before the introduction of SMTPUTF8 and that has remained so. I
don't believe turning off SMTPUTF8 in Postfix will change the behaviour
in this case.

John

On 03/03/18 22:10, Stephan Bosch wrote:
>
> Clearly, the relevant specifications don't allow UTF-8 in the local
part
> without email address internationalization (EAI), which Dovecot does not
> support nor announce yet (although that should be mended somewhat soon).
> My preferred fix for now would be to reject addresses like that, which
> would maybe still mean that this message is rejected entirely.
> Alternatively, that address at least needs to be skipped, ignored, or
> modified: all of which aren't very nice things to do. The alternative
is
> forwarding this violation to other systems, which is seldom acceptable
> either.
>
Would accepting UTF8 local part in the From header have any negative
consequences for Dovecot? If it does not I would accept it, based on the
fact that it maintains greater interoperability with other systems.
John

* A.L.E.C <alec at alec.pl>:
> On 03/02/2018 03:32 PM, Ralf Hildebrandt wrote:
> > The address causing the error is:
> > 
> > From: =?utf-8?Q?Dorit_M=C3=BCller?= <d.m?ller at JOMEC.de>
> > 
> > Note the "umlaut" in the email address... :)
> 
> This is about SMTPUTF8 (RFC6531). Looks like your only option is to disable
> smtputf8_enable in Postfix config.
It IS already disabled.

-- 
Ralf Hildebrandt
  Gesch?ftsbereich IT | Abteilung Netzwerk
  Charit? - Universit?tsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebrandt at charite.de | https://www.charite.de
	    
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20180305/2397adb5/attachment.sig>