dovecot - Apr 2007 - Feature Request: content-filter / MOVE scripts

Since your "1.1 plans" announcement somehow turned into everybody 
handing in a wish list . . . thought I'd start one by itself - since I 
expect a few people to have opinions on this matter.

I'm aware of a plug-in functionality in Dovecot, and that a plug-in 
exists for dspam processing - but my review of the plug-ins indicate 
they require compiling your own Dovecot - the plug-in setup isn't really 
suitable for packaged distros.  As a packaged software user - I really 
really really don't like compiling my own software, either for features 
or bugfixes - I want leave that to the experts.  On learning of 1.0's 
official release, I nudged the Debian package maintainers to check - and 
they were already on it and it updated.  Thanx Timo, Jaldhar, and Fabio.

I have some thoughts on an interface that would eliminate the need for 
compile-time plug-ins, but provide some major power to sysadmins.  
Basically, I want a way of telling an external program the following:
1.  A message was moved.
2.  The folder is moved from was XXX.
3.  The folder it moved to was YYY.
4.  The message moved was ZZZ.

Armed with this information, just about anybody's spam fighting learn 
scripts can be called to do their thing.

A few different ways have occurred to me:

1.  Have a dedicated "content-filter" functionality built-in to 
Dovecot.  Under this model, Dovecot.conf would include at least three 
new parameters: FILTER-FOLDER-SUFFIX, FILTER-FOLDER-IN-SCRIPT, and 
FILTER-FOLDER-OUT-SCRIPT.  Whenever a message is moved into or out of a 
folder ending in SUFFIX the appropriate SCRIPT is called, with the 
arguments above being passed.

2.  A bit more generic, less processing for Dovecot.  Dovecot.conf 
parameter ACTION-SCRIPT (hey, feel free to use other names - I'm just 
brainstorming here).  Any time any message is moved anywhere, call 
SCRIPT with the arguments.   Having never looked at Dovecot's code, I'm 
just guessing, but this probably wouldn't take much to implement - but 
might be the most performance affecting.

3.  Have a definable named-pipe to send the same information to.

4.  Have a definable TCP socket to send the same information to.

In any case, I figure it's best to prevent any message processing by 
Dovecot itself - so while it might be nice to define a list of headers 
to send to the scripts, instead of the whole message, that's probably 
not appropriate for Dovecot to focus on.  On the other hand, any of the 
above options would probably benefit from a MAXIMUM_FILTER_SIZE 
parameter to limit the maximum message size.

So what's the world got to say about this?  (More importantly, how does 
TIMO feel about this?) ;).

-- 
Daniel

Daniel L. Miller wrote:
> I have some thoughts on an interface that would eliminate the need for 
> compile-time plug-ins, but provide some major power to sysadmins.  
> Basically, I want a way of telling an external program the following:
> 1.  A message was moved.
> 2.  The folder is moved from was XXX.
> 3.  The folder it moved to was YYY.
> 4.  The message moved was ZZZ.
This sounds like the sort of job DBus 
(http://www.freedesktop.org/wiki/Software_2fdbus) is well suited for. 
Announcing events that other people may want to react to.
> Armed with this information, just about anybody's spam fighting learn 
> scripts can be called to do their thing.
Add some more events, and people can get all the logging they want, too...

I've occasionally considered how one would go about opening up a program
like
Dovecot to dbus, and there are several approaches.

1) Minimal exposure:  just send a few event signals with relevant information.
2) Clean API:  Expose many hooks into the system for event output and external 
program interfacing.
3) Open slather:  Break the app up into several DBus objects, talking amongst 
themselves.

Of course, option3 would somewhat cease to be Dovecot, but with its strong, 
clear APIs Dovecot would be an excellent starting point.  With fully exposed 
internals, you could achieve all sorts of interesting extensions and the like.

but this is a job for someone with more spare time :)

--
Curtis Maloney
cmaloney at cardgate.net

On Thu, 2007-04-19 at 08:07 -0700, Daniel L. Miller
wrote:
> 1.  Have a dedicated "content-filter" functionality built-in to 
> Dovecot.  Under this model, Dovecot.conf would include at least three 
> new parameters: FILTER-FOLDER-SUFFIX, FILTER-FOLDER-IN-SCRIPT, and 
> FILTER-FOLDER-OUT-SCRIPT.  Whenever a message is moved into or out of a 
> folder ending in SUFFIX the appropriate SCRIPT is called, with the 
> arguments above being passed.
This has been suggested several times earlier also. I think I was
planning on implementing this as some kind of a plugin ("ext-events"?)
which would have its own configuration file, something like:

copy {
  # * being the default for both source/dest
  source_mailbox = *
  dest_mailbox = Spam
  # command gets the mail from stdin
  command = /usr/bin/teach-spam.sh
}
copy {
  source_mailbox = Spam
  command = /usr/bin/teach-ham.sh
}

There could be other events as well each having their own filter
settings. Instead of command there could also be a "pipe" or a
"socket"
setting.
> In any case, I figure it's best to prevent any message processing by 
> Dovecot itself - so while it might be nice to define a list of headers 
> to send to the scripts, instead of the whole message, that's probably 
> not appropriate for Dovecot to focus on.  On the other hand, any of the 
> above options would probably benefit from a MAXIMUM_FILTER_SIZE 
> parameter to limit the maximum message size.
It's easy to get a list of specific headers, so that could be a setting
too.

But as for when this plugin in general gets implemented, I don't know.
It's not really a priority for me right now. If someone wants to try
implementing it, mail-log plugin is a great place to start since it
already contains all the hooking code. You'd just need to make it read a
config file (see src/auth/db-sql.c for example), check if an event
matches configured events and then call the configured command instead
of just logging the event.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20070514/28e61f35/attachment.bin>