Amelia A. Lewis
2003-Oct-05 22:09 UTC
[Dovecot] Line length limitation in dovecot leads to out-of-control process?
Heylas! I've been copying mailboxes from one dovecot server (on an alpha running debian testing, 0.99.10-9) to another (on an i686 running debian unstable, 0.99.10-9). I encountered a weird problem, which I seem to have resolved by moving a message with a To: line from hell out of the mailbox. Both machines are running maildir. They're not exactly identical, but they're fairly similar, apart from differences in distribution and architecture; they're running the same version of dovecot, supposedly. Specifically, when I copied this folder over to the new location, dovecot went into CPU-feeding frenzy (up to 99.7% of CPU), and stayed there. If I killed the client and tried to look at the same mailbox using a different client (mutt instead of sylpheed), without killing the imap processor on the server, the indexes would be locked, mutt would fall back to reading headers instead, and then would send its imap process into CPU-feeding frenzy when it got to one particular message. After some experimentation, I found the message. Once I moved it out of cur, I could access the mailbox without problems. The odd thing is that I can look at the message on the original machine, but on the machine it was copied to, its mere existence makes the mailbox inaccessible. After a little more exploration (grep ^To name-of-message | wc, on both machines) I seem to have found the issue: the original message has a To: line that is 16504 characters long. In the copy, it's 8192 characters long, and ends in the middle of a <email at address> (line in copy ends: user name <emai, chopped right there). I suspect that it is the *latter* that's causing dovecot to die a messy death. Or, actually, to turn into a very *rude* process that *doesn't* die. It's a really stupid trick to stuff that much into a header line (it's from helixcode in 2000, btw, notifying people that the promised CD is late), and perfectly reasonable to truncate (even to truncate to 1000 characters, per 282[12]). However, I kinda thing dovecot shouldn't choke on the results of truncation, which is what I think is the problem. Amy! -- Amelia A. Lewis amyzing {at} talsever.com I have spent nights with matches and knives, leaning over ledges, only two flights up. Cutting my heart, burning my soul. Nothing left to hold. Nothing left, but blood and fire. -- Indigo Girls
Amelia A. Lewis
2003-Oct-05 22:53 UTC
[Dovecot] Line length limitation in dovecot leads to out-of-control process?
A follow-up: The messages are roughly the same length on both machines. +2 on the copy-to machine (added LFs, I guess). It turns out that on copy, the to-line-from-hell is truncated, then continued (but without a space at the front of the "added" line, and truncation in mid-word), lather-rinse-repeat (original is 16504 long, so that's two by 8192 and a trailer). Problems therefore could be because the line ends abruptly, or because the two lines introduced are not valid header lines. Amy! -- Amelia A. Lewis amyzing {at} talsever.com I have spent nights with matches and knives, leaning over ledges, only two flights up. Cutting my heart, burning my soul. Nothing left to hold. Nothing left, but blood and fire. -- Indigo Girls
Timo Sirainen
2003-Oct-09 00:44 UTC
[Dovecot] Line length limitation in dovecot leads to out-of-control process?
On Mon, 2003-10-06 at 01:09, Amelia A. Lewis wrote:> After a little more exploration (grep ^To name-of-message | wc, on both > machines) I seem to have found the issue: the original message has a To: > line that is 16504 characters long. In the copy, it's 8192 characters > long, and ends in the middle of a <email at address> (line in copy ends: > user name <emai, chopped right there). I suspect that it is the > *latter* that's causing dovecot to die a messy death. Or, actually, to > turn into a very *rude* process that *doesn't* die.Yes, it happens when the header name is larger than 8192 bytes. Dovecot wants to read more data, but the buffer is already full and it tries again.. Actually there is code to handle this, but it has a bug somewhere. I'll fix this and check that other parsers don't have the same problem.