dovecot - Oct 2004 - Index corruption

I'm getting pretty constant index corruption on (admittedly large) 
Maildir folders.

I'm running Thunderbird 0.7.3 and Mutt as my IMAP clients (on FreeBSD).  
Server is Dovecot 0.99.10.9
configured thusly:
    ./configure --disable-ipv6 CC=gcc-3.3 CXX=g++-3.3 
--prefix=/opt/dovecot --without-pop3d
on a Solaris 2.8 box.  Maildirs are NFS-mounted, but the index files are 
on local UFS filesystem:
    default_mail_env = maildir:~/Maildir:INDEX=/export/01/imapd/%u
MTA is Postfix, LDA is procmail delivering to several Maildir folders.

There will be 2 (or more) dovecot instances running for this user (me) - 
several from Thunderbird and one from the BIFF client (icewm).  Turning 
off the BIFF client doesn't seem to help.

Symptoms:  when using Thunderbird, and deleting a msg or moving it to 
another folder, Thunderbird will sometimes not display the next unread 
message, but just sit there.  Changing folders works, but changing back 
to the previous folder (or quitting/restarting Thunderbird) only shows 
the first N messages (N is about 500, in a Maildir of about 6000, and it 
is the SAME email that is "last" every time.)  This happens with 
multiple folders, but (usually) only folders that are delivered to by 
procmail (this may be related to the problem or a coincidence in that 
only those folders have unread email!)

Accessing the mailboxes using MUTT via IMAP shows the same truncated 
message list.

(I'm the only user using dovecot as I want Maildir, the other users have 
uw-imap, so dovecot is listening on port 4343)

Closing all imap clients, then removing the .index.* files from the 
appropriate /export/01/imapd/ directory, then restarting the imap client 
causes the missing messages to reappear (after a suitable wait for the 
index files to be recreated).

It really looks like locking or truncation issues in the index files....

Any clues? Debugging hints?  This is happening several times per day so 
I can get lots o' debug data!

Greg.

On 18.10.2004, at 03:22, Gregory Bond wrote:
> I'm getting pretty constant index corruption on (admittedly large) 
> Maildir folders.
I'd suggest grabbing 1.0-test49 from http://dovecot.org/test/ and 
trying if it works better.

Indexing code (and pretty much everything else) has been rewritten in 
1.0-test releases and it's quite stable nowadays. I've noticed some 
maildir problems with stress testing (which I'll try to get around to 
fixing sometimes soon) but with normal use it works fine.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20041018/47254fd8/attachment-0001.bin>

Timo Sirainen wrote:
> I'd suggest grabbing 1.0-test49 from http://dovecot.org/test/ and 
> trying if it works better.

Wow, reply at 3am!

I can't get the imapd from this release to start up.  No matter what I 
do, I'm getting this error:

    auth(default): We couldn't drop root group privileges (wanted=1, 
gid=0, egid=0)

even before it asks for username.

I've tried starting from inetd, starting from from dovecot, setting 
first_valid_gid = 0, 1 or 2


Looking at the logic in lib/restrict-access.c I'm not sure why or how 
this can be failing.  gid=1 so setgid() must have succeeded at the top 
of restrict_access_by_env():
        if (gid != 0 && (gid != getgid() || gid != getegid())) {
                if (setgid(gid) != 0)

and uid must be != 0, else this test fails:
        if (setuid(0) == 0) {
                        if (uid == 0)
                                i_fatal("Running as root isn't
permitted");
                        i_fatal("We couldn't drop root
privileges");

so how does this test:
        if (getgid() == 0 || getegid() == 0 || setgid(0) == 0) {
ever get to be true?