Hi,
I noticed that dovecot's MySQL driver doesn't play well with UTF-8
encoded strings. I presume this issue has been around for a while. However, it
still fails with the current release candidate 2.2.16-rc1.
I have been using the sieve extdata plugin with a MySQL-backed dict to retrieve
a folder name that is then passed to fileinto:
fileinto :create "${extdata.folder_name_drafts}";
Whenever this dict returned a string with non-7byte characters, a line such as
the following popped up in sieve.log. (This example query should return
"Entw?rfe".)
error: folder name specified for fileinto command is not utf-8: Entw?rfe.
Although my MySQL installation is set up to use UTF-8 as the default charset on
every level (config, database, table and field), and the character_set_* runtime
variables all yield the value "utf8", it still seems that the mysql
client library must be instructed to actually use UTF-8 explicitly. Adding the
following statement to driver_mysql_connect() fixes the issue for me:
mysql_options(db->mysql, MYSQL_SET_CHARSET_NAME, "utf8");
I checked this against mysql 5.5.41.
Regards,
Felix
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mysql_utf8.patch
Type: application/octet-stream
Size: 559 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20150311/552337e8/attachment.obj>
On 11 Mar 2015, at 21:31, Felix Zandanel <felix at zandanel.me> wrote:> > Although my MySQL installation is set up to use UTF-8 as the default charset on every level (config, database, table and field), and the character_set_* runtime variables all yield the value "utf8", it still seems that the mysql client library must be instructed to actually use UTF-8 explicitly. Adding the following statement to driver_mysql_connect() fixes the issue for me: > > mysql_options(db->mysql, MYSQL_SET_CHARSET_NAME, "utf8");I think you can also add to /etc/my.cnf : [client] default-character-set = utf8
> Am 12.03.2015 um 14:55 schrieb Timo Sirainen <tss at iki.fi>: > > On 11 Mar 2015, at 21:31, Felix Zandanel <felix at zandanel.me> wrote: >> >> Although my MySQL installation is set up to use UTF-8 as the default charset on every level (config, database, table and field), and the character_set_* runtime variables all yield the value "utf8", it still seems that the mysql client library must be instructed to actually use UTF-8 explicitly. Adding the following statement to driver_mysql_connect() fixes the issue for me: >> >> mysql_options(db->mysql, MYSQL_SET_CHARSET_NAME, "utf8"); > > I think you can also add to /etc/my.cnf : > > [client] > default-character-set = utf8Sorry for the late reply. You were so right, that simple line did the trick. My fault, I didn't read the whole charset documentation of MySQL. It's a shame that UTF-8 isn't the default setting. Anyway, as dovecot's internals expect all input strings to be UTF-8, wouldn't it be useful to enforce UTF-8 in the database drivers? Using anything else than ASCII / UTF-8 for dovecot's MySQL connections doesn't really make sense, I think. Also, a "default-character-set = utf8" line in my.cnf is a system wide configuration, which might break other software interacting with MySQL?in theory. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://dovecot.org/pipermail/dovecot/attachments/20150323/303b42a9/attachment.sig>