dovecot - May 2007 - UW-IMAP style mbox snarfing from /var/mail/ to ~/mbox

http://dovecot.org/patches/mbox-snarf-plugin.c

Also committed to CVS HEAD.

Usage:

mail_location = mbox:~/mail:INBOX=/var/mail/%u

protocol imap {
  mail_plugins = mbox_snarf
}
protocol pop3 {
  mail_plugins = mbox_snarf
}

plugin {
  mbox_snarf = ~/mbox
}

If ~/mbox doesn't exist, nothing special is done.

If ~/mbox exists, it's treated as INBOX and mails are moved
from /var/mail/%u whenever the mailbox is synchronized. If all the
messages couldn't be copied, for example because of out of quota, then
none of the messages are copied. I suppose that could be a problem. It
might be possible to fix that by changing:

		if (mailbox_copy(dest_trans, mail, 0, NULL, NULL) < 0) {
			if (!mail->expunged) {
				ret = -1;
				break;
			}
		}

to just:

		if (mailbox_copy(dest_trans, mail, 0, NULL, NULL) < 0) {
			if (!mail->expunged)
				break;
		}

But I'm not sure about that. Try and tell me. :)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20070517/aab7edcf/attachment.bin>

Timo Sirainen wrote:
> http://dovecot.org/patches/mbox-snarf-plugin.c
Hi,

I just tried to setup this plugin on a Debian Etch machine but it 
crashes the Dovecot daemons.

Debian Etch uses the sourcecode of Dovecot 1.0rc15 + some patches.

I rebuilt the Dovecot packages -common, -imapd and -pop3d using

apt-get install apt-build
apt-build build-source dovecot-common

and installed them. Then I compiled the .c file mentioned above
inside the sourcedir created by apt-build by interrupting apt-build,
then issuing these commands:


cd /var/cache/apt-build/build/dovecot-1.0.rc15/
wget http://dovecot.org/patches/mbox-snarf-plugin.c

export DOVECOT=/var/cache/apt-build/build/dovecot-1.0.rc15

gcc -fPIC -shared -Wall -I$DOVECOT -I$DOVECOT/src/lib
-I$DOVECOT/src/lib-storage -I$DOVECOT/src/lib-mail
-I$DOVECOT/src/lib-imap -DHAVE_CONFIG_H      mbox-snarf-plugin.c -o
mbox_snarf_plugin.so

The compiled file mbox_snarf_plugin.so was copied to 
/usr/lib/dovecot/modules/lib10_mbox_snarf_plugin.so
with symlinks inside /usr/lib/dovecot/modules/imap and 
/usr/lib/dovecot/modules/pop3.

I setup /etc/dovecot/dovecot.conf according to your posting and 
restarted Dovecot. But accessing Dovecot with Thunderbird produces this:
> May 29 16:01:09 bach dovecot: Killed with signal 15
> May 29 16:01:10 bach dovecot: Dovecot v1.0.rc15 starting up
> May 29 16:01:24 bach dovecot: imap-login: Login: user=<mail>,
method=plain, rip=x.x.x.x, lip=x.x.x.x, TLS
> May 29 16:01:24 bach dovecot: child 28786 (imap) killed with signal 11
> May 29 16:01:25 bach dovecot: imap-login: Login: user=<mail>,
method=plain, rip=x.x.x.x, lip=x.x.x.x, TLS
> May 29 16:01:25 bach dovecot: child 28791 (imap) killed with signal 11
> May 29 16:01:25 bach dovecot: imap-login: Login: user=<mail>,
method=plain, rip=x.x.x.x, lip=x.x.x.x, TLS
> May 29 16:01:25 bach dovecot: child 28795 (imap) killed with signal 11
> May 29 16:01:25 bach dovecot: imap-login: Login: user=<mail>,
method=plain, rip=x.x.x.x, lip=x.x.x.x, TLS
> May 29 16:01:25 bach dovecot: child 28806 (imap) killed with signal 11
I suppose I did something wrong when compiling the module, but what?

Greetings, Bernd Kuhls

Timo Sirainen wrote:
> mail_location = mbox:~/mail:INBOX=/var/mail/%u
Hi,

I am using Dovecot 1.0.0 (will soon update to 1.0.1) using your patch.

This is my mail_location setting:
> mail_location =
mbox:~/.imap_mail:INBOX=/var/mail/%u:INDEX=/var/mail/indexes/%u
The index for $HOME/.imap_mail/inbox is however created in
$HOME/.imap_mail/.imap/inbox/, while the indexes for other mbox
files are created in /var/mail/indexes/%u, like its supposed to be.

Is this a bug or a misconfiguration on my part?

Greetings, Bernd Kuhls

Timo Sirainen wrote:
> http://dovecot.org/patches/mbox-snarf-plugin.c
Hi,

small update from an end-user of this plugin:

generally it works like expected, but some problems I mentioned are not 
solved yet. Here are my main config settings:

mail_location = 
mbox:~/.imap_mail:INBOX=/var/mail/%u:INDEX=/var/mail/indexes/%u

mbox_snarf = ~/.imap_mail/inbox

- The index for $HOME/.imap_mail/inbox is created in
   $HOME/.imap_mail/.imap/inbox/, while the indexes for other mbox files
   are created in /var/mail/indexes/%u, like its supposed to be. How to
   disable creating index files in $HOME altogether without using links?

- if /var/mail/$user does not exist, syslog is cluttered with these
   messages for each POP3/IMAP login:

   Jun 10 22:33:33 server dovecot: IMAP(someuser): stat() failed with
   mbox file /var/mail/someuser: No such file or directory

- the biggest problem is the following scenario:
   - /var/mail/$user does not exist
   - Exim can not deliver a mail to $HOME/.imap_mail/ so a second router
     comes into action and delivers the mail to /var/mail/$user
   - /var/mail/$user now only contains the mail, without the IMAP headers
     Dovecot normally writes to a mbox file
   - the user logs into the account and Dovecot tries to snarf the mail
   - if snarfing is sucessful Dovecot must delete the mail from
     /var/mail/$user, but this fails due to the missing IMAP headers
   - when the user logs into the account the next time the mail is
     snarfed again, this can be repeated endlessly
   - the only situation where the snarf plugin works like expected is
     when /var/mail/$user contains the IMAP headers

To circumvent this problem I created a cronjob running once per minute 
which creates /var/mail/$user for $HOME/* with this content:
> From MAILER_DAEMON  Sat Apr 21 19:48:15 2007
> Date: Sat, 21 Apr 2007 19:48:15 +0200
> From: Mail System Internal Data <MAILER-DAEMON at server>
> Subject: DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA
> Message-ID: <1177177695 at server>
> X-IMAP: 1177169799 0000003411 NonJunk Junk $Forwarded
> 
> Status: RO
> 
> This text is part of the internal format of your mail folder, and is not
> a real message.  It is created automatically by the mail system software.
> If deleted, important folder data will be lost, and it will be re-created
> with the data reset to initial values.
This solution is annoying because it does not really fix the problem.
Can you help me with this problem? I am using Dovecot 1.0.0, might 
updating to 1.0.1 help fixing some of the problems mentioned above?

Greetings, Bernd

Bernd Kuhls wrote:
> I am using Dovecot 1.0.0, might updating to 1.0.1 help fixing some of
> the problems mentioned above?
Hi,

updated to Dovecot 1.0.3, but all the mentioned problems still exist.

Greetings, Bernd