dovecot - Oct 2003 - Line length limitation in dovecot leads to out-of-control process?

Heylas!

I've been copying mailboxes from one dovecot server (on an alpha running
debian testing, 0.99.10-9) to another (on an i686 running debian
unstable, 0.99.10-9).  I encountered a weird problem, which I seem to
have resolved by moving a message with a To: line from hell out of the
mailbox.  Both machines are running maildir.  They're not exactly
identical, but they're fairly similar, apart from differences in
distribution and architecture; they're running the same version of
dovecot, supposedly.

Specifically, when I copied this folder over to the new location,
dovecot went into CPU-feeding frenzy (up to 99.7% of CPU), and stayed
there.  If I killed the client and tried to look at the same mailbox
using a different client (mutt instead of sylpheed), without killing the
imap processor on the server, the indexes would be locked, mutt would
fall back to reading headers instead, and then would send its imap
process into CPU-feeding frenzy when it got to one particular message.

After some experimentation, I found the message.  Once I moved it out of
cur, I could access the mailbox without problems.  The odd thing is that
I can look at the message on the original machine, but on the machine it
was copied to, its mere existence makes the mailbox inaccessible.

After a little more exploration (grep ^To name-of-message | wc, on both
machines) I seem to have found the issue: the original message has a To:
line that is 16504 characters long.  In the copy, it's 8192 characters
long, and ends in the middle of a <email at address> (line in copy ends:
user name <emai, chopped right there).  I suspect that it is the
*latter* that's causing dovecot to die a messy death.  Or, actually, to
turn into a very *rude* process that *doesn't* die.

It's a really stupid trick to stuff that much into a header line (it's
from helixcode in 2000, btw, notifying people that the promised CD is
late), and perfectly reasonable to truncate (even to truncate to 1000
characters, per 282[12]).  However, I kinda thing dovecot shouldn't
choke on the results of truncation, which is what I think is the
problem.

Amy!
-- 
Amelia A. Lewis                    amyzing {at} talsever.com
I have spent nights with matches and knives, leaning over ledges, only 
two flights up.  Cutting my heart, burning my soul.  Nothing left to
hold.  Nothing left, but blood and fire.
                -- Indigo Girls

A follow-up:

The messages are roughly the same length on both machines.  +2 on the
copy-to machine (added LFs, I guess).  It turns out that on copy, the
to-line-from-hell is truncated, then continued (but without a space at
the front of the "added" line, and truncation in mid-word),
lather-rinse-repeat (original is 16504 long, so that's two by 8192 and a
trailer).  Problems therefore could be because the line ends abruptly,
or because the two lines introduced are not valid header lines.

Amy!
-- 
Amelia A. Lewis                    amyzing {at} talsever.com
I have spent nights with matches and knives, leaning over ledges, only 
two flights up.  Cutting my heart, burning my soul.  Nothing left to
hold.  Nothing left, but blood and fire.
                -- Indigo Girls

On Mon, 2003-10-06 at 01:09, Amelia A. Lewis wrote:
> After a little more exploration (grep ^To name-of-message | wc, on both
> machines) I seem to have found the issue: the original message has a To:
> line that is 16504 characters long.  In the copy, it's 8192 characters
> long, and ends in the middle of a <email at address> (line in copy
ends:
> user name <emai, chopped right there).  I suspect that it is the
> *latter* that's causing dovecot to die a messy death.  Or, actually, to
> turn into a very *rude* process that *doesn't* die.
Yes, it happens when the header name is larger than 8192 bytes. Dovecot
wants to read more data, but the buffer is already full and it tries
again.. Actually there is code to handle this, but it has a bug
somewhere. I'll fix this and check that other parsers don't have the
same problem.