Hi, Thank you for your reply. I hope this bug to be fixed soon. :) Thank you. Tachibana ----- Original Message -----> On Mon, Jul 20, 2020 at 09:43:12 -0400, Josef 'Jeff' Sipek wrote: > > On Mon, Jul 20, 2020 at 20:24:13 +0900, TACHIBANA Masashi wrote: > ... > > Thanks for the report. I reproduced it locally, but I'm not sure what is > > causing it yet. > > Alright, it is actually a parsing bug. The problem seems to be when the > code encounters a "name <address>" format where the name is a valid address. > This accidentally makes the parser stop parsing - making any additional > addresses ignored. > > E.g., > > test at example.com <test at example.com>, Name <other at example.com> > > The parser stops after the first address's name thinking it is just a bare > address (without a name). That is, after the first loop iterating while > parsing, the "cursor" is at the first space and not at the comma. > > /- parsing should stop here > v > test at example.com <test at example.com>, Name <other at example.com> > ^ > `- parsing stops here > > This is obviously wrong. I'll have to dig in more to figure out what it > will take to fix this. > > Thanks again, > > Jeff. > > -- > FreeBSD 12.1 >-- TACHIBANA Masashi QUALITIA CO., LTD. mailto:tachibana at qualitia.co.jp ?????????? https://www.qualitia.co.jp/
On Tue, Jul 21, 2020 at 08:18:21 +0900, TACHIBANA Masashi wrote:> Hi, > > Thank you for your reply. > > I hope this bug to be fixed soon. :)It is a tricky one because as far as I can tell, the From header is invalid. So, we have to figure out the best way to handle such input. The input is invalid because '@' is not allowed by the RFC to be in the name portion of an email address without quoting. In other words, these are ok: test at example.com <test at example.com> Test <test at example.com> "test at example.com" <test at example.com> but this one is bad: test at example.com <test at example.com> In your case, the name is encoded but not quoted so even though the email contains: =?UTF-8?B?dXNlcjJAZXhhbXBsZS5jb20=?= <user2 at example.com> the parser sees: user2 at example.com <user2 at example.com> Which is invalid. Dovecot parsing needs to be improved, but as I already mentioned it is an improvement to handling of malformed input. So, as a workaround, if you control the software/system that generates these emails, try to change it so it doesn't generate headers that are invalid. As far as I can tell, the parser in Dovecot handles those just fine. Jeff.> Thank you. > > Tachibana > > ----- Original Message ----- > > On Mon, Jul 20, 2020 at 09:43:12 -0400, Josef 'Jeff' Sipek wrote: > > > On Mon, Jul 20, 2020 at 20:24:13 +0900, TACHIBANA Masashi wrote: > > ... > > > Thanks for the report. I reproduced it locally, but I'm not sure what is > > > causing it yet. > > > > Alright, it is actually a parsing bug. The problem seems to be when the > > code encounters a "name <address>" format where the name is a valid address. > > This accidentally makes the parser stop parsing - making any additional > > addresses ignored. > > > > E.g., > > > > test at example.com <test at example.com>, Name <other at example.com> > > > > The parser stops after the first address's name thinking it is just a bare > > address (without a name). That is, after the first loop iterating while > > parsing, the "cursor" is at the first space and not at the comma. > > > > /- parsing should stop here > > v > > test at example.com <test at example.com>, Name <other at example.com> > > ^ > > `- parsing stops here > > > > This is obviously wrong. I'll have to dig in more to figure out what it > > will take to fix this. > > > > Thanks again, > > > > Jeff. > > > > -- > > FreeBSD 12.1 > > > -- > TACHIBANA Masashi QUALITIA CO., LTD. > mailto:tachibana at qualitia.co.jp > > ?????????? > https://www.qualitia.co.jp/ > >-- Hegh QaQ law' quvHa'ghach QaQ puS
Hi, Thank you Jeff san. It seems to be able to work fine in IMAP, without FTS. But I hope FTS to find addresses. And I reported this to the developers of email solutions which adding such MIMEd display name. But for older mails or another email softwares, I hope this problems to be solved in Dovecot with FTS. Thank you. ----- Original Message -----> On Tue, Jul 21, 2020 at 08:18:21 +0900, TACHIBANA Masashi wrote: > > Hi, > > > > Thank you for your reply. > > > > I hope this bug to be fixed soon. :) > > It is a tricky one because as far as I can tell, the From header is invalid. > So, we have to figure out the best way to handle such input. > > The input is invalid because '@' is not allowed by the RFC to be in the name > portion of an email address without quoting. In other words, these are ok: > > test at example.com > <test at example.com> > Test <test at example.com> > "test at example.com" <test at example.com> > > but this one is bad: > > test at example.com <test at example.com> > > In your case, the name is encoded but not quoted so even though the email > contains: > > =?UTF-8?B?dXNlcjJAZXhhbXBsZS5jb20=?= <user2 at example.com> > > the parser sees: > > user2 at example.com <user2 at example.com> > > Which is invalid. > > Dovecot parsing needs to be improved, but as I already mentioned it is an > improvement to handling of malformed input. So, as a workaround, if you > control the software/system that generates these emails, try to change it so > it doesn't generate headers that are invalid. As far as I can tell, the > parser in Dovecot handles those just fine. > > Jeff. > > > Thank you. > > > > Tachibana > > > > ----- Original Message ----- > > > On Mon, Jul 20, 2020 at 09:43:12 -0400, Josef 'Jeff' Sipek wrote: > > > > On Mon, Jul 20, 2020 at 20:24:13 +0900, TACHIBANA Masashi wrote: > > > ... > > > > Thanks for the report. I reproduced it locally, but I'm not sure what is > > > > causing it yet. > > > > > > Alright, it is actually a parsing bug. The problem seems to be when the > > > code encounters a "name <address>" format where the name is a valid address. > > > This accidentally makes the parser stop parsing - making any additional > > > addresses ignored. > > > > > > E.g., > > > > > > test at example.com <test at example.com>, Name <other at example.com> > > > > > > The parser stops after the first address's name thinking it is just a bare > > > address (without a name). That is, after the first loop iterating while > > > parsing, the "cursor" is at the first space and not at the comma. > > > > > > /- parsing should stop here > > > v > > > test at example.com <test at example.com>, Name <other at example.com> > > > ^ > > > `- parsing stops here > > > > > > This is obviously wrong. I'll have to dig in more to figure out what it > > > will take to fix this. > > > > > > Thanks again, > > > > > > Jeff. > > > > > > -- > > > FreeBSD 12.1 > > > > > -- > > TACHIBANA Masashi QUALITIA CO., LTD. > > mailto:tachibana at qualitia.co.jp > > > > ?????????? > > https://www.qualitia.co.jp/ > > > > > > -- > Hegh QaQ law' > quvHa'ghach QaQ puS >-- TACHIBANA Masashi QUALITIA CO., LTD. mailto:tachibana at qualitia.co.jp ?????????? https://www.qualitia.co.jp/