dovecot - Dec 2005 - writing an MDA, help me avoid corrupted indexes

Hello.

First, I want to thank those responsible for Dovecot IMAP as it was a 
real treat to implement in comparison to my mostly-failed attempts with 
Cyrus.  The passwd-file authentication was exactly what I needed as I 
really didn't want to have to deal with PAM or LDAP but where I am 
setting up a server with virtual accounts.

I'm using Postfix as an MTA, and it quite handily passes the mail off to 
SpamAssassin.  From there I need an MDA to dump any mails detected as 
spam into a common spam folder, discard any mails with viruses (antivir) 
and send a notification to me, and then drop good mail into the 
appropriate maildir directory for the virtual users to get via both POP3 
and IMAP.

I could conceivably use procmail to do that MDA, and I've done similar 
things with procmail in the past, but my growing experience with 
procmail indicates that procmail is too ferocious for what simple work I 
need done, and too difficult to make work right in a virtual-user 
environment.  So I've written my own little MDA with a few lines of 
shell that will take the mail from Postfix+SpamAssassin and ultimately 
deposit mails in the maildirs.

So far so good, except that I keep seeing corrupt indexes... probably 
because I'm not touching the indexes at all with the MDA.  My guess is 
that in order for the MDA to properly deal with maildir it needs to 
adjust the indexes after depositing new mail.  Okay.  So can someone 
either give me a pointer to the proper documentation on this, including 
the index file formats, or perhaps just explain it to me?  Maybe I'm 
missing a larger picture of maildir handling?

I've glanced through http://www.dovecot.org/doc/index.txt but I notice 
that the index files are not ASCII, so I'm not sure where to start at 
understanding that documentation.

Any and all help would be appreciated.

Thanks,

Lee.

Lee Howard wrote:
> Hello.
> 
> I could conceivably use procmail to do that MDA, and I've done similar 
> things with procmail in the past, but my growing experience with 
> procmail indicates that procmail is too ferocious for what simple work I 
> need done, and too difficult to make work right in a virtual-user 
> environment.  So I've written my own little MDA with a few lines of 
> shell that will take the mail from Postfix+SpamAssassin and ultimately 
> deposit mails in the maildirs.
Have you checked out Dovecot's own LDA?  It includes index updating as,
really,
its primary purpose, but also Sieve filtering.  If you just want the LDA to 
filter on headers set by your spam/virus scanners, it's ample.
> So far so good, except that I keep seeing corrupt indexes... probably 
> because I'm not touching the indexes at all with the MDA.  My guess is 
> that in order for the MDA to properly deal with maildir it needs to 
> adjust the indexes after depositing new mail.  Okay.  So can someone 
> either give me a pointer to the proper documentation on this, including 
> the index file formats, or perhaps just explain it to me?  Maybe I'm 
> missing a larger picture of maildir handling?
> 
My understanding was that Dovecot fixes corrupt indexes as a matter of course, 
and isn't too bothered with it.  What level do you have your logging set to?

Do you, by chance, have your mail partition mounted 'noatime'?
> I've glanced through http://www.dovecot.org/doc/index.txt but I notice 
> that the index files are not ASCII, so I'm not sure where to start at 
> understanding that documentation.
> 
> Any and all help would be appreciated.
As a further note, have you checked out the patches that allow Postfix to use 
the Dovecot auth daemon, so you can have SMTP AUTH without resorting to the 
horror that is Cyrus SASL?

--
Curtis Maloney
cmaloney@cardgate.net