-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, I have seen the following assertion fails on my own mail server (indexer-worker): Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE) However, when charset_to_utf8_try returns FALSE (e.g. iconv() got EINVAL or EILSEQ), the for loop in charset_to_utf8 may end earlier, and there may be additional pending characters, and the assertion doesn't seem always hold in the error case, perhaps we should assert that pos is non-zero in the error case? Cheers, - -- Xin LI <delphij at delphij.net> https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.1.2 (FreeBSD) iQIcBAEBCgAGBQJVVjO/AAoJEJW2GBstM+nsbKUP/29fYdZArCiUSStK0hCtRvC3 o4L35zq4iP4WUry7VqNtdMg2Uiythei9l0JokNOJAy2ZDktJ3ns9CArXHi/pJE7/ V8374Iu9ygb4tzDvcNY/353SFkN+Sw4pJzSGJTelVT8nHxahQMtN7E0u26l54zNx p1H8rgt2nVA8eRvKL9MrhudN3eSI2+Jq4HX+dR4/Tu4qLkSU1nqi8sopQxNXzW4C 4XG4haSbTtXLu0H1MGa8y5pmwrFmZEluNOBm2SOww+UqeteD/MwE/pgA7g+cQx8l /t3FWzSe4C9HROV1VGW5V5HDM7cDvNJZ+nxQGI6L6qInkCBqQx/muIb9nIe7dhro tiHIuX4h8X2zaga+iD+B9fkf4QpDUc11WYE+0DUArb2cNTTAgeTXQ68+g0Q+UC8s OxUVDA4KwOSzkpEifDgvVCdgPG7U87SGe9NlMGjSZvAIEPsww16A3f2IaM4fhFa2 lVuTh1wq8phdWXLbCbZ0F36hAyxVbbwC5Xyj4Ygy3QL4RiBC7CJJ8I4f1lRLnoFz 1Wg0/M7psXBLVN7vx2MKtK5Rj1OvMOKmL4VpkhKcnp2z/Y9I5Qq2CP4C5uOv3DLJ JjoCsnVDpWJKQo6EPleb9OxVJF+NmjHd63zimXbuUyDJ7AJCsx/0m9mJZ5QYf+jU U3OqESCU5e5q++QLd2Kl =ksoA -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, On 5/15/15 10:58, Xin Li wrote:> Hi, > > I have seen the following assertion fails on my own mail server > (indexer-worker): > > Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion > failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE)I've managed to narrow down the issue to one mail, and can reliably trigger the assertion with 'doveadm index -u user INBOX' with only that email. Maybe we should reset the iconv descriptor after each full conversion? i.e.: diff -r 91bbd6753689 src/lib-mail/message-decoder.c - --- a/src/lib-mail/message-decoder.c Thu May 14 23:49:12 2015 +0300 +++ b/src/lib-mail/message-decoder.c Fri May 15 22:52:41 2015 -0700 @@ -347,6 +347,8 @@ sizeof(ctx->translation_buf)); memcpy(ctx->translation_buf, data + pos, ctx->translation_size); + } else { + charset_to_utf8_reset(ctx->charset_trans); } output->data = ctx->buf2->data; output->size = ctx->buf2->used; Cheers, -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVVuYpAAoJEJW2GBstM+nsrr8P+wZRjznaTJuAE7eaKlmO5Zba m9bUHsHh/rZdjfTCfUiwYwU66+EI8O2WEgr0sXynMrRhRWp5BVWMKGTl2loJ/oNL O3qXrGoVmAJoWmnd+ZQ49s+zSJUgXk9rJneN1i4erId6vjN1pcWXfrLbmeKhreN8 EmOMDO8WtV/XBq20UerKu+kGnatC8tNzpTQ5PCTnMc9z1nvNytE/NxM3Gli3wTRG 8nhZY8fS+y6guJfi68RZK6Th1W6jKQQW8+sNPlNbCRPxDb5delXLGq58DnQVk4mM Y6VxzQF4ItTU5h+ikNOaY7v4LoC3uydUdhPkMVVb5JVH+wx9en7PefhV4dH2MNjJ a8dJ4gvsYNBLZ1r3QLFF0NqJMhPXtidKLy7iQkBW0Z1LqmQrEpwZpLVGrdDtWLpA E4HkOmTMEyIbTuK5hG5zNyoF2+XLHSCAELZBHxJAxoHhwakm3n4MhJJao+aJho8+ DbgbGMFk4dDgvOAD0gqANSrIfpZkWGAzUiNiMP015Trg4bQm7CSztuvM6+xhr8EE zzvYjbeCAl3A1+u2q2UFaw8Ze60kD345FyRmkkxLxgJY3r32fSaU+1VvazlbIFKa zq4R2aospgIIxwVQOH3drqGgXpxEe54QG++3IQxmZyNCh3qocb+FnatVupHIJzX/ vvqgdswgAsXP04duqKKn =QaAX -----END PGP SIGNATURE-----
On 16 May 2015, at 09:39, Xin Li <delphij at delphij.net> wrote:> > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Hi, > > On 5/15/15 10:58, Xin Li wrote: >> Hi, >> >> I have seen the following assertion fails on my own mail server >> (indexer-worker): >> >> Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion >> failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE) > > I've managed to narrow down the issue to one mail, and can reliably > trigger the assertion with 'doveadm index -u user INBOX' with only that > email.Can you send me some mail that triggers the crash? I'm having trouble reproducing.> Maybe we should reset the iconv descriptor after each full conversion? > i.e.: > > diff -r 91bbd6753689 src/lib-mail/message-decoder.c > - --- a/src/lib-mail/message-decoder.c Thu May 14 23:49:12 2015 +0300 > +++ b/src/lib-mail/message-decoder.c Fri May 15 22:52:41 2015 -0700 > @@ -347,6 +347,8 @@ > sizeof(ctx->translation_buf)); > memcpy(ctx->translation_buf, data + pos, > ctx->translation_size); > + } else { > + charset_to_utf8_reset(ctx->charset_trans); > }I think the fix probably belongs to charset-iconv.c