dovecot - Sep 2008 - Panic in Dovecot 1.1.3: index-mail.c: line 1091: assertion failed: (!mail->data.destroying_stream)

Dovecot 1.1.3
Solaris 10
SPARC (Sun Fire T1000)
Compiled with Sun Studio 12 compilers.
Maildir on NFS
Indexes on local disk (UFS).
'dovecot -n' output attached.


IMAP process crashes for certain (many, but not all) users when
accessing certain folders (in the example below, in crashes when
accessing my INBOX, about 1700 mails). I could access other
mailboxes without problems. And a simple telnet to the imap port
followed by a login works fine.

Erased the coredumps (was filling up the system too quickly) so I
can't really produce a backtrace right now (had to back down to 1.0.13
again even though that one has another core-dump-generating problem -
but that is atleast limited to only one specific user so far). Going
to set up a separate test server to test things more without disturbing
our normal operations...


I started with an empty INDEX directory to make sure there weren't
problems with old corrupted indexes.


Output from syslog:

Sep  9 12:34:44 ifm.liu.se dovecot: [ID 107833 mail.info] imap-login:
Login: user=<peter>, method=GSSAPI, rip=130.236.160.102,
lip=130.236.160.6, secured

Sep  9 12:34:44 ifm.liu.se dovecot: [ID 107833 mail.crit] Panic:
IMAP(peter): file index-mail.c: line 1091: assertion failed:
(!mail->data.destroying_stream)

Sep  9 12:34:44 ifm.liu.se dovecot: [ID 107833 mail.error] IMAP(peter):
Raw backtrace: 0x1000b5188 -> 0x1000605ac -> 0x1000605e4 -> 0x100060734
-> 0x1000657b8 -> 0x100070470 -> 0x1000204e0 -> 0x100020868 ->
0x100016e78 -> 0x10001dab8 -> 0x10001debc -> 0x10001e0a4 ->
0x1000bf4b4
-> 0x1000bea68 -> 0x10002933c -> 0x100014c9c

Sep  9 12:34:48 ifm.liu.se dovecot: [ID 107833 mail.info] auth(default):
client out: OK 1       user=peter

Sep  9 12:34:48 ifm.liu.se dovecot: [ID 107833 mail.info] auth(default):
passwd(peter,130.236.160.102): lookup

Sep  9 12:34:48 ifm.liu.se dovecot: [ID 107833 mail.info] auth(default):
master out: USER       241     peter   system_user=peter       uid=103
gid=10  home=/home/peter

Sep  9 12:34:48 ifm.liu.se dovecot: [ID 107833 mail.info] imap-login:
Login: user=<peter>, method=GSSAPI, rip=130.236.160.102,
lip=130.236.160.6, secured

Sep  9 12:34:48 ifm.liu.se dovecot: [ID 107833 mail.crit] Panic:
IMAP(peter): file index-mail.c: line 1091: assertion failed:
(!mail->data.destroying_stream)

Sep  9 12:34:48 ifm.liu.se dovecot: [ID 107833 mail.error] IMAP(peter):
Raw backtrace: 0x1000b5188 -> 0x1000605ac -> 0x1000605e4 -> 0x100060734
-> 0x1000657b8 -> 0x100070470 -> 0x1000204e0 -> 0x100020868 ->
0x100016e78 -> 0x10001dab8 -> 0x10001debc -> 0x10001e0a4 ->
0x1000bf4b4
-> 0x1000bea68 -> 0x10002933c -> 0x100014c9c


- Peter
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dovecot-n.out
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20080909/20dcad70/attachment-0002.pl>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 185 bytes
Desc: OpenPGP digital signature
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20080909/20dcad70/attachment-0002.bin>

On Tue, 2008-09-09 at 13:23 +0200, Peter Eriksson wrote:
> Maildir on NFS
This is the first time I've heard this happening with maildir. It's
always been with mboxes before.
> IMAP process crashes for certain (many, but not all) users when
> accessing certain folders (in the example below, in crashes when
> accessing my INBOX, about 1700 mails). I could access other
> mailboxes without problems. And a simple telnet to the imap port
> followed by a login works fine.
Any idea what the users were doing when it crashed? Was it just opening
the mailbox or opening some mail or deleting/copying mails?
> Erased the coredumps (was filling up the system too quickly) so I
> can't really produce a backtrace right now (had to back down to 1.0.13
> again even though that one has another core-dump-generating problem -
> but that is atleast limited to only one specific user so far). Going
> to set up a separate test server to test things more without disturbing
> our normal operations...
Some kind of a way to reproduce this would be helpful. Or I guess in
your case even a backtrace, since all the previous ones have been with
COPY command and mbox.
> mmap_disable: yes
> mail_nfs_index: yes
BTW. these can be "no" if you're storing indexes locally.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20080909/8ede08d8/attachment-0002.bin>

Timo Sirainen skrev:
>
>> IMAP process crashes for certain (many, but not all) users when
>> accessing certain folders (in the example below, in crashes when
>> accessing my INBOX, about 1700 mails). I could access other
>> mailboxes without problems. And a simple telnet to the imap port
>> followed by a login works fine.
>>     
>
> Any idea what the users were doing when it crashed? Was it just opening
> the mailbox or opening some mail or deleting/copying mails?
>
>   
I have set up the isolated test server now so that I can test this a bit 
better. It crashes when I start
up a fresh Dovecot instance, start up Thunderbird and then click on the 
INBOX on that server
(I rsync-copied the whole tree (my mails) to a separare, local, 
directory so I can test things without
disturbing my normal mailbox).

I have noticed one more thing now since I sent that mail though - I 
noticed that I didn't have
debug info in the binaries (originally compiled with "-fast -m64" for 
high machine-specific
optimizations) so I recompiled with "-g" added to the compiler flags
and
retested.
Same problem.

Then I removed all optimization flags (only compiled with "-g -m64"
and
now it stopped crashing...
So my current thesis is that it is something in that code that optimizes 
wrong (or when optimized
exposes some bug) for some reason.

Going to try some variants of optimization flags and compilers (have 
some patches for the
Sun Studio 12 compilers that I'm going to apply too) and see if I can 
narrow things down
a bit more.
>> Erased the coredumps (was filling up the system too quickly) so I
>> can't really produce a backtrace right now (had to back down to
1.0.13
>> again even though that one has another core-dump-generating problem -
>> but that is atleast limited to only one specific user so far). Going
>> to set up a separate test server to test things more without disturbing
>> our normal operations...
>>     
>
> Some kind of a way to reproduce this would be helpful. Or I guess in
> your case even a backtrace, since all the previous ones have been with
> COPY command and mbox.
>
>   
I'll send a backtrace in a little while.
>> mmap_disable: yes
>> mail_nfs_index: yes
>>     
>
> BTW. these can be "no" if you're storing indexes locally.
>   
Yeah, I know. I was just a leftover from the original server (I 
originally stored the indexes in the Maildir folder,
but decided to store the locally while testing 1.1.3 so I wouldn't 
disturb the 1.0.13 generated ones in case I
had to go back (which I did)).

- Peter