Op 7-3-2018 om 15:40 schreef Stephan Bosch:> > > Op 7-3-2018 om 13:46 schreef Philipp Berger: >> I wrapped the LDA command in a script. I can see that Postfix passes >> "@@mydomain.tld" as the -d argument, without quotes. >> I then adapted the script to specifically replace this address with >> "@"@mydomain.tld, but this results in the following error message by >> Dovecot: >> >> ???? auth: Info: userdb(?): Username character disallowed by >> auth_username_chars: 0x22 (username: "@"@mydomain.tld) >> >> So what would be the appropriate quoting/setting for this address? > > Adding " to auth_username_chars would help. > > But why are you trying to accept such incredibly horrible > email-addresses/usernames? >Also, -d is used to provide the username. For -d, you could leave out the double quotes and provide the full e-mail address--including the required quotes--to the -r argument. It all depends on what your userdb expects. Regards, Stephan.>> Am 06-Mar-18 um 15:08 schrieb Stephan Bosch: >>> >>> Op 6-3-2018 om 14:34 schreef Philipp Berger: >>>> I upgraded to Dovecot 2.3.0.1 as advised, but it still seems >>>> broken. In >>>> the Postfix log I now see: >>>> >>>> Mar? 6 13:49:03 myhost amavis[7165]: (07165-10) K00VtLRHdrYw FWD from >>>> <admin at mydomain.tld> -> <"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from >>>> MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37 >>>> Mar? 6 13:49:03 myhost amavis[7165]: (07165-10) Passed CLEAN >>>> {RelayedInbound}, [someIp]:11439 [someIp] <admin at mydomain.tld> -> >>>> <"@"@mydomain.tld>, Queue-ID: D27792DA167C, Message-ID: >>>> <22b95756-e95e-86cf-219c-3b603f7587c8 at mydomain.tld>, mail_id: >>>> K00VtLRHdrYw, Hits: -3, size: 7472, queued_as: B8CA22DA1B37, 1863 ms >>>> Mar? 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C: >>>> to=<@@mydomain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=2, >>>> delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0 from >>>> MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37) >>>> Mar? 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37: >>>> to=<@@mydomain.tld>, relay=dovecot, delay=0.2, delays=0.11/0/0/0.09, >>>> dsn=5.1.1, status=bounced (user unknown) >>>> Mar? 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80: >>>> message-id=<20180306124903.EA37E2DA1F80 at mydomain.tld> >>>> Mar? 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37: sender >>>> non-delivery notification: EA37E2DA1F80 >>>> Mar? 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80: from=<>, >>>> size=9830, nrcpt=1 (queue active) >>>> Mar? 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37: removed >>>> Mar? 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80: >>>> to=<admin at mydomain.tld>, relay=dovecot, delay=0.07, >>>> delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command line usage >>>> error. Command output: lda: Fatal: Invalid -f parameter: Missing >>>> domain ) >>>> >>>> I can see in my MySQL Log that Dovecot queried at the same time >>>> with %n >>>> = "" (empty) and %d = "@mydomain.tld". So now the "@" is dropped >>>> entirely. >>>> >>>> What does the "Fatal: Invalid -f parameter: Missing domain" tell >>>> us? Do >>>> I need to change the way Postfix calls Dovecot? Add more quotes? :D >>> Can you check what Postfix is feeding to Dovecot exactly (i.e., >>> whether it is properly escaped)? Maybe wrap dovecot-lda in some shell >>> script and see what is passed. >>> >>> Regards, >>> >>> Stephan. >>> >>>> Kind regards, >>>> Philipp >>>> >>>> Am 04-Mar-18 um 21:12 schrieb Stephan Bosch: >>>>> Op 3/1/2018 om 2:07 PM schreef Philipp Berger: >>>>>> Dear all, >>>>>> >>>>>> I have a working setup with Postfix + Dovecot, storing users in a >>>>>> MySQL >>>>>> table. >>>>>> >>>>>> I ran into problems setting up and using a mail address like >>>>>> a"@"b at mydomain.tld, which by RFC should be valid, but leads to >>>>>> problems >>>>>> in Dovecot. From my debugging, I can see that on the lookup, Dovecot >>>>>> replaces %n with "a" (quotes added by me) and %d with >>>>>> "b at mydomain.tld" >>>>>> (quotes added by me). So the original quotes are gone, and the >>>>>> domain/user split is incorrect. >>>>>> >>>>>> I am not sure of Postfix passes the address without quotes to >>>>>> Dovecot or >>>>>> whether Dovecot can not handle quotes correctly, but I was hoping >>>>>> that >>>>>> someone with more insight into Dovecot could help me out here. >>>>>> >>>>>> System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot >>>>>> 2.2.27-3+deb9u1 >>>>>> Postfix master.cf for Dovecot: >>>>>> dovecot unix? -?????? n?????? n?????? -?????? - pipe >>>>>> ??? flags=DRhu user=virtual:virtual >>>>>> argv=/usr/lib/dovecot/dovecot-lda -f >>>>>> ${sender} -d ${user}@${nexthop} >>>>>> >>>>>> I am grateful for any hints :) >>>>>> >>>>>> Thank you in advance, kind regards, >>>>> Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address >>>>> parsing. >>>>> Dovecot v2.3 should do a better job. I am still not sure if this >>>>> can be >>>>> fixed well for Dovecot v2.2. >>>>> >>>>> Regards, >>>>> >>>>> Stephan. >>>> >> >
Interesting, the documentation and the build-in help of dovecot-lda do
not mention -r exists :)
case 'r':
/* final recipient address */
if (smtp_address_parse_path(ctx.pool, optarg,
SMTP_ADDRESS_PARSE_FLAG_ALLOW_LOCALPART |
SMTP_ADDRESS_PARSE_FLAG_BRACKETS_OPTIONAL,
&final_rcpt_to, &errstr) < 0) {
i_fatal_status(EX_USAGE,
"Invalid -a parameter: %s", errstr);
Btw: Should the error message not read "Invalid -r parameter: %s"
instead?
I think the problem is the different handling in Postfix and Dovecot:
Postfix seems to automatically strip the quotes out of the username, at
least for SQL lookup (but does the splitting correctly, which Dovecot
2.3 does NOT).
Dovecot does not strip quotes.
What is the recommended (i.e. endorsed by its developers) way for
Dovecot to deal with Mail addresses containing Quotation marks? Strip
them out? Keep them? Since they are disabled by default
(auth_username_chars), it seems like stripping, but then the splitting
of local part and domain does not work correctly.
Am I really the first one to play around with this? Does it make sense
to prepare patches for example for fixing the splitting?
On a side note: Has support for UTF8 in local parts already landed in
Dovecot? I could not find up-to-date information on this topic.
Kind regards,
Philipp
Am 07-Mar-18 um 15:50 schrieb Stephan Bosch:>
>
> Op 7-3-2018 om 15:40 schreef Stephan Bosch:
>>
>>
>> Op 7-3-2018 om 13:46 schreef Philipp Berger:
>>> I wrapped the LDA command in a script. I can see that Postfix
passes
>>> "@@mydomain.tld" as the -d argument, without quotes.
>>> I then adapted the script to specifically replace this address with
>>> "@"@mydomain.tld, but this results in the following error
message by
>>> Dovecot:
>>>
>>> ???? auth: Info: userdb(?): Username character disallowed by
>>> auth_username_chars: 0x22 (username: "@"@mydomain.tld)
>>>
>>> So what would be the appropriate quoting/setting for this address?
>>
>> Adding " to auth_username_chars would help.
>>
>> But why are you trying to accept such incredibly horrible
>> email-addresses/usernames?
>>
> Also, -d is used to provide the username. For -d, you could leave out
> the double quotes and provide the full e-mail address--including the
> required quotes--to the -r argument.
>
> It all depends on what your userdb expects.
>
> Regards,
>
> Stephan.
>
>>> Am 06-Mar-18 um 15:08 schrieb Stephan Bosch:
>>>>
>>>> Op 6-3-2018 om 14:34 schreef Philipp Berger:
>>>>> I upgraded to Dovecot 2.3.0.1 as advised, but it still
seems
>>>>> broken. In
>>>>> the Postfix log I now see:
>>>>>
>>>>> Mar? 6 13:49:03 myhost amavis[7165]: (07165-10)
K00VtLRHdrYw FWD from
>>>>> <admin at mydomain.tld> ->
<"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from
>>>>> MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as
B8CA22DA1B37
>>>>> Mar? 6 13:49:03 myhost amavis[7165]: (07165-10) Passed
CLEAN
>>>>> {RelayedInbound}, [someIp]:11439 [someIp] <admin at
mydomain.tld> ->
>>>>> <"@"@mydomain.tld>, Queue-ID: D27792DA167C,
Message-ID:
>>>>> <22b95756-e95e-86cf-219c-3b603f7587c8 at
mydomain.tld>, mail_id:
>>>>> K00VtLRHdrYw, Hits: -3, size: 7472, queued_as:
B8CA22DA1B37, 1863 ms
>>>>> Mar? 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C:
>>>>> to=<@@mydomain.tld>,
relay=127.0.0.1[127.0.0.1]:10024, delay=2,
>>>>> delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0
from
>>>>> MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as
B8CA22DA1B37)
>>>>> Mar? 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37:
>>>>> to=<@@mydomain.tld>, relay=dovecot, delay=0.2,
delays=0.11/0/0/0.09,
>>>>> dsn=5.1.1, status=bounced (user unknown)
>>>>> Mar? 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80:
>>>>> message-id=<20180306124903.EA37E2DA1F80 at
mydomain.tld>
>>>>> Mar? 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37:
sender
>>>>> non-delivery notification: EA37E2DA1F80
>>>>> Mar? 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80:
from=<>,
>>>>> size=9830, nrcpt=1 (queue active)
>>>>> Mar? 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37:
removed
>>>>> Mar? 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80:
>>>>> to=<admin at mydomain.tld>, relay=dovecot,
delay=0.07,
>>>>> delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command
line usage
>>>>> error. Command output: lda: Fatal: Invalid -f parameter:
Missing
>>>>> domain )
>>>>>
>>>>> I can see in my MySQL Log that Dovecot queried at the same
time
>>>>> with %n
>>>>> = "" (empty) and %d = "@mydomain.tld".
So now the "@" is dropped
>>>>> entirely.
>>>>>
>>>>> What does the "Fatal: Invalid -f parameter: Missing
domain" tell
>>>>> us? Do
>>>>> I need to change the way Postfix calls Dovecot? Add more
quotes? :D
>>>> Can you check what Postfix is feeding to Dovecot exactly (i.e.,
>>>> whether it is properly escaped)? Maybe wrap dovecot-lda in some
shell
>>>> script and see what is passed.
>>>>
>>>> Regards,
>>>>
>>>> Stephan.
>>>>
>>>>> Kind regards,
>>>>> Philipp
>>>>>
>>>>> Am 04-Mar-18 um 21:12 schrieb Stephan Bosch:
>>>>>> Op 3/1/2018 om 2:07 PM schreef Philipp Berger:
>>>>>>> Dear all,
>>>>>>>
>>>>>>> I have a working setup with Postfix + Dovecot,
storing users in a
>>>>>>> MySQL
>>>>>>> table.
>>>>>>>
>>>>>>> I ran into problems setting up and using a mail
address like
>>>>>>> a"@"b at mydomain.tld, which by RFC
should be valid, but leads to
>>>>>>> problems
>>>>>>> in Dovecot. From my debugging, I can see that on
the lookup,
>>>>>>> Dovecot
>>>>>>> replaces %n with "a" (quotes added by me)
and %d with
>>>>>>> "b at mydomain.tld"
>>>>>>> (quotes added by me). So the original quotes are
gone, and the
>>>>>>> domain/user split is incorrect.
>>>>>>>
>>>>>>> I am not sure of Postfix passes the address without
quotes to
>>>>>>> Dovecot or
>>>>>>> whether Dovecot can not handle quotes correctly,
but I was
>>>>>>> hoping that
>>>>>>> someone with more insight into Dovecot could help
me out here.
>>>>>>>
>>>>>>> System Info: Debian 9, Postifx 3.1.6-0+deb9u1,
Dovecot
>>>>>>> 2.2.27-3+deb9u1
>>>>>>> Postfix master.cf for Dovecot:
>>>>>>> dovecot unix? -?????? n?????? n?????? -?????? -
pipe
>>>>>>> ??? flags=DRhu user=virtual:virtual
>>>>>>> argv=/usr/lib/dovecot/dovecot-lda -f
>>>>>>> ${sender} -d ${user}@${nexthop}
>>>>>>>
>>>>>>> I am grateful for any hints :)
>>>>>>>
>>>>>>> Thank you in advance, kind regards,
>>>>>> Yeah, Dovecot v2.2 is doing some funky stuff with SMTP
address
>>>>>> parsing.
>>>>>> Dovecot v2.3 should do a better job. I am still not
sure if this
>>>>>> can be
>>>>>> fixed well for Dovecot v2.2.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Stephan.
>>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20180307/8b95ce89/attachment-0001.html>
Op 3/7/2018 om 6:28 PM schreef Philipp Berger:> Interesting, the documentation and the build-in help of dovecot-lda do > not mention -r exists :) > > case 'r': > > /* final recipient address */ > > if (smtp_address_parse_path(ctx.pool, optarg, > > SMTP_ADDRESS_PARSE_FLAG_ALLOW_LOCALPART | > > SMTP_ADDRESS_PARSE_FLAG_BRACKETS_OPTIONAL, > > &final_rcpt_to, &errstr) < 0) { > > i_fatal_status(EX_USAGE, > > "Invalid -a parameter: %s", errstr); > > > Btw: Should the error message not read "Invalid -r parameter: %s" instead?Yes, both problems fixed. Thanks. Full documentation is here: https://wiki2.dovecot.org/LDA> > I think the problem is the different handling in Postfix and Dovecot: > Postfix seems to automatically strip the quotes out of the username, > at least for SQL lookup (but does the splitting correctly, which > Dovecot 2.3 does NOT).Can you specify which split is wrong in your opinion? If the MTA strips necessary quotes before it is passed to dovecot- lda, Dovecot will reject the address. That is not wrong.> Dovecot does not strip quotes.That depends on what Dovecot is working operating on. When it is provided as a username (-d), it will not strip quotes. If it is an email address (-a, -r, -f), it will strip quotes if they're unnecessary. Internally, it uses an unquoted, split representation (localpart and domain separately). Once it needs to compose the split addresses back into a string, quotes are added only when needed.> What is the recommended (i.e. endorsed by its developers) way for > Dovecot to deal with Mail addresses containing Quotation marks? Strip > them out? Keep them? Since they are disabled by default > (auth_username_chars), it seems like stripping, but then the splitting > of local part and domain does not work correctly. > Am I really the first one to play around with this? Does it make sense > to prepare patches for example for fixing the splitting?Only if it is broken or breaks compatibility for existing setups in a bad way (e.g. Dovecot allows some address syntax violations commonly seen for some Asian ISPs).> On a side note: Has support for UTF8 in local parts already landed in > Dovecot? I could not find up-to-date information on this topic.Not yet. We will be tending to that pretty soon though. Regards, Stephan.> > Am 07-Mar-18 um 15:50 schrieb Stephan Bosch: >> >> >> Op 7-3-2018 om 15:40 schreef Stephan Bosch: >>> >>> >>> Op 7-3-2018 om 13:46 schreef Philipp Berger: >>>> I wrapped the LDA command in a script. I can see that Postfix passes >>>> "@@mydomain.tld" as the -d argument, without quotes. >>>> I then adapted the script to specifically replace this address with >>>> "@"@mydomain.tld, but this results in the following error message by >>>> Dovecot: >>>> >>>> ???? auth: Info: userdb(?): Username character disallowed by >>>> auth_username_chars: 0x22 (username: "@"@mydomain.tld) >>>> >>>> So what would be the appropriate quoting/setting for this address? >>> >>> Adding " to auth_username_chars would help. >>> >>> But why are you trying to accept such incredibly horrible >>> email-addresses/usernames? >>> >> Also, -d is used to provide the username. For -d, you could leave out >> the double quotes and provide the full e-mail address--including the >> required quotes--to the -r argument. >> >> It all depends on what your userdb expects. >> >> Regards, >> >> Stephan. >> >>>> Am 06-Mar-18 um 15:08 schrieb Stephan Bosch: >>>>> >>>>> Op 6-3-2018 om 14:34 schreef Philipp Berger: >>>>>> I upgraded to Dovecot 2.3.0.1 as advised, but it still seems >>>>>> broken. In >>>>>> the Postfix log I now see: >>>>>> >>>>>> Mar? 6 13:49:03 myhost amavis[7165]: (07165-10) K00VtLRHdrYw FWD >>>>>> from >>>>>> <admin at mydomain.tld> -> <"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from >>>>>> MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37 >>>>>> Mar? 6 13:49:03 myhost amavis[7165]: (07165-10) Passed CLEAN >>>>>> {RelayedInbound}, [someIp]:11439 [someIp] <admin at mydomain.tld> -> >>>>>> <"@"@mydomain.tld>, Queue-ID: D27792DA167C, Message-ID: >>>>>> <22b95756-e95e-86cf-219c-3b603f7587c8 at mydomain.tld>, mail_id: >>>>>> K00VtLRHdrYw, Hits: -3, size: 7472, queued_as: B8CA22DA1B37, 1863 ms >>>>>> Mar? 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C: >>>>>> to=<@@mydomain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=2, >>>>>> delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0 from >>>>>> MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37) >>>>>> Mar? 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37: >>>>>> to=<@@mydomain.tld>, relay=dovecot, delay=0.2, delays=0.11/0/0/0.09, >>>>>> dsn=5.1.1, status=bounced (user unknown) >>>>>> Mar? 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80: >>>>>> message-id=<20180306124903.EA37E2DA1F80 at mydomain.tld> >>>>>> Mar? 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37: sender >>>>>> non-delivery notification: EA37E2DA1F80 >>>>>> Mar? 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80: from=<>, >>>>>> size=9830, nrcpt=1 (queue active) >>>>>> Mar? 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37: removed >>>>>> Mar? 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80: >>>>>> to=<admin at mydomain.tld>, relay=dovecot, delay=0.07, >>>>>> delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command line usage >>>>>> error. Command output: lda: Fatal: Invalid -f parameter: Missing >>>>>> domain ) >>>>>> >>>>>> I can see in my MySQL Log that Dovecot queried at the same time >>>>>> with %n >>>>>> = "" (empty) and %d = "@mydomain.tld". So now the "@" is dropped >>>>>> entirely. >>>>>> >>>>>> What does the "Fatal: Invalid -f parameter: Missing domain" tell >>>>>> us? Do >>>>>> I need to change the way Postfix calls Dovecot? Add more quotes? :D >>>>> Can you check what Postfix is feeding to Dovecot exactly (i.e., >>>>> whether it is properly escaped)? Maybe wrap dovecot-lda in some shell >>>>> script and see what is passed. >>>>> >>>>> Regards, >>>>> >>>>> Stephan. >>>>> >>>>>> Kind regards, >>>>>> Philipp >>>>>> >>>>>> Am 04-Mar-18 um 21:12 schrieb Stephan Bosch: >>>>>>> Op 3/1/2018 om 2:07 PM schreef Philipp Berger: >>>>>>>> Dear all, >>>>>>>> >>>>>>>> I have a working setup with Postfix + Dovecot, storing users in a >>>>>>>> MySQL >>>>>>>> table. >>>>>>>> >>>>>>>> I ran into problems setting up and using a mail address like >>>>>>>> a"@"b at mydomain.tld, which by RFC should be valid, but leads to >>>>>>>> problems >>>>>>>> in Dovecot. From my debugging, I can see that on the lookup, >>>>>>>> Dovecot >>>>>>>> replaces %n with "a" (quotes added by me) and %d with >>>>>>>> "b at mydomain.tld" >>>>>>>> (quotes added by me). So the original quotes are gone, and the >>>>>>>> domain/user split is incorrect. >>>>>>>> >>>>>>>> I am not sure of Postfix passes the address without quotes to >>>>>>>> Dovecot or >>>>>>>> whether Dovecot can not handle quotes correctly, but I was >>>>>>>> hoping that >>>>>>>> someone with more insight into Dovecot could help me out here. >>>>>>>> >>>>>>>> System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot >>>>>>>> 2.2.27-3+deb9u1 >>>>>>>> Postfix master.cf for Dovecot: >>>>>>>> dovecot unix? -?????? n?????? n?????? -?????? - pipe >>>>>>>> ??? flags=DRhu user=virtual:virtual >>>>>>>> argv=/usr/lib/dovecot/dovecot-lda -f >>>>>>>> ${sender} -d ${user}@${nexthop} >>>>>>>> >>>>>>>> I am grateful for any hints :) >>>>>>>> >>>>>>>> Thank you in advance, kind regards, >>>>>>> Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address >>>>>>> parsing. >>>>>>> Dovecot v2.3 should do a better job. I am still not sure if this >>>>>>> can be >>>>>>> fixed well for Dovecot v2.2. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Stephan. >>>>>> >>>> >>> >> >