dovecot - Feb 2015 - Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd)

Hello.

I wrote about this three weeks ago but got no answer. I'm going to
officially "forward" the Debian bug this time, with all the details.

The test case is just 840 bytes long. Please give it a try.

---------- Forwarded message ----------
From: Santiago Vila <sanvila at unex.es>
To: submit at bugs.debian.org
Date: Fri, 23 Jan 2015 22:32:28 +0100 (CET)
Subject: dovecot-imapd: corrupts mailbox after trying to retrieve it

Package: dovecot-imapd
Version: 1:2.2.13-11
Severity: serious

The following mbox folder, when put in $HOME/mail, becomes corrupted after
trying to retrieve it with fetchmail.

The problem may be reproduced by using the same machine as server and client:

* Put "inbox-b" in $HOME/mail

* Put this in $HOME/.fetchmailrc

server localhost proto imap port 143:
 user "someuser"
 pass "thepassword"
  
* Retrieve email using this command line:

fetchmail -a localhost --folder inbox-b -m "true"


Note: By looking at the "true" above it is clear that whatever
fetchmail does with the message is not important at all.


You will see something like this:

12 messages for someuser at localhost (folder inbox-b).
reading message someuser at localhost:1 of 12 (171 header octets) (3 body
octets) flushed
reading message someuser at localhost:2 of 12 (245 header octets) (3 body
octets) flushed
reading message someuser at localhost:3 of 12 (245 header octets) (3 body
octets) flushed
reading message someuser at localhost:4 of 12 (245 header octets) (3 body
octets) flushed
reading message someuser at localhost:5 of 12 (245 header octets) (3 body
octets) flushed
reading message someuser at localhost:6 of 12 (171 header octets) (3 body
octets) flushed
reading message someuser at localhost:7 of 12 (171 header octets) (3 body
octets) flushed
reading message someuser at localhost:8 of 12 (245 header octets) (3 body
octets) flushed
reading message someuser at localhost:9 of 12 (245 header octets) (3 body
octets) flushed
reading message someuser at localhost:10 of 12 (245 header octets) (3 body
octets) flushed
reading message someuser at localhost:11 of 12 (245 header octets) (3 body
octets) flushed
reading message someuser at localhost:12 of 12 (273 header octets)fetchmail:
incorrect header line found - see manpage for bad-header option
 not flushed


And in fact "inbox-b" in the server is now like this:

[...]
>From root at example.com  Tue Jan 13 10:18:20 2015
rstuvwxyzabcdefghijklmnopqrstuvwxyz at example.com
To: a at example.com
Subject: a
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Message-Id: <20150113091737.B5ADA5F8B1 at example.com>
Date: Tue, 13 Jan 2015 10:17:25 +0100 (CET)
X-UID: 16035
Status: O

a


Note how the From: line has been truncated from its original state.


I have been suffering from this problem for months. At first I believed
it was some misbehaving procmail/formail recipe I had on the server,
but that's not the case as this example shows.

Thanks.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: inbox-b.gz
Type: application/gzip
Size: 840 bytes
Desc: 
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20150214/77f6361c/attachment.bin>

On 14 Feb 2015, at 16:23, Santiago Vila <sanvila at unex.es> wrote:
> I wrote about this three weeks ago but got no answer. I'm going to
> officially "forward" the Debian bug this time, with all the
details.
> 
> The test case is just 840 bytes long. Please give it a try.
..
> Package: dovecot-imapd
> Version: 1:2.2.13-11
> Severity: serious
I can't reproduce with latest Dovecot hg. But just in case it's still
not fixed, there are two important things:

1) Send your doveconf -n output, since there are some settings that can affect
this

2) rm -rf ~/mail/.imap/inbox-b before testing to make sure indexes don't
cause this problem.
> The following mbox folder, when put in $HOME/mail, becomes corrupted after
> trying to retrieve it with fetchmail.
> 
> The problem may be reproduced by using the same machine as server and
client:
> 
> * Put "inbox-b" in $HOME/mail
> 
> * Put this in $HOME/.fetchmailrc
> 
> server localhost proto imap port 143:
> user "someuser"
> pass "thepassword"
> 
> * Retrieve email using this command line:
> 
> fetchmail -a localhost --folder inbox-b -m "true"
> 
> 
> Note: By looking at the "true" above it is clear that whatever
> fetchmail does with the message is not important at all.
> 
> 
> You will see something like this:
> 
> 12 messages for someuser at localhost (folder inbox-b).
> reading message someuser at localhost:1 of 12 (171 header octets) (3 body
octets) flushed
> reading message someuser at localhost:2 of 12 (245 header octets) (3 body
octets) flushed
> reading message someuser at localhost:3 of 12 (245 header octets) (3 body
octets) flushed
> reading message someuser at localhost:4 of 12 (245 header octets) (3 body
octets) flushed
> reading message someuser at localhost:5 of 12 (245 header octets) (3 body
octets) flushed
> reading message someuser at localhost:6 of 12 (171 header octets) (3 body
octets) flushed
> reading message someuser at localhost:7 of 12 (171 header octets) (3 body
octets) flushed
> reading message someuser at localhost:8 of 12 (245 header octets) (3 body
octets) flushed
> reading message someuser at localhost:9 of 12 (245 header octets) (3 body
octets) flushed
> reading message someuser at localhost:10 of 12 (245 header octets) (3 body
octets) flushed
> reading message someuser at localhost:11 of 12 (245 header octets) (3 body
octets) flushed
> reading message someuser at localhost:12 of 12 (273 header
octets)fetchmail: incorrect header line found - see manpage for bad-header
option
> not flushed
> 
> 
> And in fact "inbox-b" in the server is now like this:
> 
> [...]
>> From root at example.com  Tue Jan 13 10:18:20 2015
> rstuvwxyzabcdefghijklmnopqrstuvwxyz at example.com
> To: a at example.com
> Subject: a
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
> Message-Id: <20150113091737.B5ADA5F8B1 at example.com>
> Date: Tue, 13 Jan 2015 10:17:25 +0100 (CET)
> X-UID: 16035
> Status: O
> 
> a
> 
> 
> Note how the From: line has been truncated from its original state.
> 
> 
> I have been suffering from this problem for months. At first I believed
> it was some misbehaving procmail/formail recipe I had on the server,
> but that's not the case as this example shows.
> 
> Thanks.<inbox-b.gz>

On Sun, 15 Feb 2015, Timo Sirainen wrote:
> On 14 Feb 2015, at 16:23, Santiago Vila <sanvila at unex.es> wrote:
> 
> > I wrote about this three weeks ago but got no answer. I'm going to
> > officially "forward" the Debian bug this time, with all the
details.
> > 
> > The test case is just 840 bytes long. Please give it a try.
[ Small correction: It's really 3873 bvtes long, uncompressed ].
> ..
> > Package: dovecot-imapd
> > Version: 1:2.2.13-11
> > Severity: serious
> 
> I can't reproduce with latest Dovecot hg.
In such case we would love to know what is the commit that fixed this,
so that we can apply it to the 2.2.13 version in Debian. We have frozen
the distribution as we are about to release jessie as Debian 8, so
no new upstream releases are allowed anymore.
> But just in case it's still not fixed, there are two important things:
> 
> 1) Send your doveconf -n output, since there are some settings that can
affect this
Nothing special. The default configuration from the Debian package.
The bug may be reproduced without touching any file in /etc at all.
This is the result of "doveconf -n":

# 2.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.0 
mail_location = mbox:~/mail:INBOX=/var/mail/%u
namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix = 
}
passdb {
  driver = pam
}
protocols = " imap"
ssl = no
userdb {
  driver = passwd
}

> 2) rm -rf ~/mail/.imap/inbox-b before testing to make sure indexes
don't cause this problem.
Yes, I did that. In fact, I checked that this happens on a freshly installed
virtual machine,
i.e. no ~/mail/.imap at all.

On Sun, Feb 15, 2015 at 10:55:13AM +0200, Timo Sirainen
wrote:
> On 14 Feb 2015, at 16:23, Santiago Vila <sanvila at unex.es> wrote:
> 
> > I wrote about this three weeks ago but got no answer. I'm going to
> > officially "forward" the Debian bug this time, with all the
details.
> > 
> > The test case is just 840 bytes long. Please give it a try.
> ..
> > Package: dovecot-imapd
> > Version: 1:2.2.13-11
> > Severity: serious
> 
> I can't reproduce with latest Dovecot hg. But just in case it's
still not fixed, there are two important things:
Would you please try with 2.2.15, which is the latest released version?

I have just tested version 2.2.15 in Debian experimental, and I can
still reproduce the bug.

Thanks.