dovecot - Oct 2008 - First release (v0.1.0) of the new Sieve implementation for Dovecot v1.2

Hello Dovecot users,

Finally, after little more than a year, I finished the first release of 
the new Sieve implementation for Dovecot.

The main reason for rewriting the Sieve engine is to provide more 
reliable script execution and to provide better error messages to users 
and system administrators. Also, since the Sieve language evolves 
quickly, with new language extensions published every year, the aim is 
to provide support for quickly extending the engine with new features.

This implementation aims to comply to RFC8225 and it provides support 
for the following Sieve language extensions:

* fileinto
* reject
* envelope
* encoded-character
* vacation
* subaddress
* relational
* regex
* imap4flags
* copy
* include
* body
* variables

In comparison to the old CMU Sieve plugin, I added support for the very 
useful variables extension. However, the notify extension is notably 
missing. This will be implemented for next major release (0.2.x).

Refer to the README file for a full list of its features:

http://hg.rename-it.nl/dovecot-libsieve/raw-file/0.1.0/README

Installation
------------

The source package is available as follows:

http://www.rename-it.nl/dovecot/1.2/dovecot-1.2-sieve-0.1.0.tar.gz
http://www.rename-it.nl/dovecot/1.2/dovecot-1.2-sieve-0.1.0.tar.gz.sig

Alternatively, it can be cloned from the mercurial repository:

hg clone http://hg.rename-it.nl/dovecot-libsieve

The compilation procedure is identical to the cmusieve plugin 
(http://wiki.dovecot.org/LDA/Sieve). Don't forget to execute 
./autogen.sh first if you obtained the sources through mercurial. This 
release will not compile against Dovecot versions older than v1.2.alpha3.

The configuration is very similar to the original plugin as well. The 
most notable difference is the name of the plugin, which is simply 
'sieve' in stead of 'cmusieve'.

After compilation, the Sieve engine can be verified before deployment 
using the included test suite. This is done by executing `make test` in 
the top of the source tree. The test suite is known to finish flawlessly 
on the i386 and amd64 platforms. The test suite is not very 
comprehensive yet, as it only tests the evaluation of scripts and omits 
testing the actual execution of actions.

As this release contains about 30k effective lines of fresh code, bugs 
will eventually surface. Make sure that you provide enough information 
to reproduce the problem if you report bugs, i.e. the script and the 
mail message that caused the problem and/or relevant log entries.

Have fun testing the new Sieve plugin. Don't hesitate to notify me when 
there are problems.

Regards,

-- 
Stephan Bosch
stephan at rename-it.nl
IRC: Freenode, #dovecot, S[r]us

Hi Stephan,

I had a problem while compiling 0.1.0 libsieve with gcc3.4.
---
/usr/include/bits/fcntl2.h:51: error: call to '__open_missing_mode'
declared with attribute error: open with O_CREAT in second argument needs 3
arguments
---
I propose a patch:

$ cat dovecot-1.2-libsieve-gcc34.patch
diff -ur dovecot1.2-libsieve-v1.2.orig/src/sieve-bin/bin-common.c
dovecot1.2-libsieve-v1.2/src/sieve-bin/bin-common.c
--- dovecot1.2-libsieve-v1.2.orig/src/sieve-bin/bin-common.c    2008-10-28
21:37:16 +0000
+++ dovecot1.2-libsieve-v1.2/src/sieve-bin/bin-common.c 2008-10-28 21:24:57
+0000
@@ -116,7 +116,7 @@
        if ( strcmp(filename, "-") == 0 )
                dumpstream = o_stream_create_fd(1, 0, FALSE);
        else {
-               if ( (dfd = open(filename, O_WRONLY | O_CREAT)) < 0 ) {
+               if ( (dfd = open(filename, O_WRONLY | O_CREAT, 0660)) < 0 ) {
                        i_fatal("Failed to open dump-file for writing:
%m");
                        exit(1);
                }
diff -ur dovecot1.2-libsieve-v1.2.orig/src/testsuite/testsuite.c
dovecot1.2-libsieve-v1.2/src/testsuite/testsuite.c
--- dovecot1.2-libsieve-v1.2.orig/src/testsuite/testsuite.c     2008-10-28
21:37:16 +0000
+++ dovecot1.2-libsieve-v1.2/src/testsuite/testsuite.c  2008-10-28 21:35:18
+0000
@@ -119,7 +119,7 @@
        if ( strcmp(filename, "-") == 0 )
                dumpstream = o_stream_create_fd(1, 0, FALSE);
        else {
-               if ( (dfd = open(filename, O_WRONLY | O_CREAT)) < 0 ) {
+               if ( (dfd = open(filename, O_WRONLY | O_CREAT, 0660)) < 0 ) {
                        i_fatal("Failed to open dump-file for writing:
%m");
                        exit(1);
                }

-- 
   Sergey Ivanov.

----- "Stephan Bosch" <stephan at rename-it.nl> wrote:
> Hello Dovecot users,
> 
> Finally, after little more than a year, I finished the first release
> of 
> the new Sieve implementation for Dovecot.
[skip]
> Have fun testing the new Sieve plugin. Don't hesitate to notify me
> when 
> there are problems.
> 
> Regards,
> 
> -- 
> Stephan Bosch
> stephan at rename-it.nl
> IRC: Freenode, #dovecot, S[r]us

sergey ivanov schreef:
> Hi Stephan,
> Thank you for quick fix. I've built it (from changeset
> 646:747107b816dc) and it looks like working with dovecot-1.2.
> I'm doing some testing before put it into Sisyphus repository
> (http://en.altlinux.org).
> What I saw - it does not recognize lines like:
> ---
>   sieve = /var/sievescripts/%n/sieve
> ---
> as cmusieve did. But it works ok with default .dovecot.sieve files in
> user home directories
Hmm, I'll look into that. This is not intended and would be a bug.
> Then, it reports
> ---
> error: failed to store into mailbox 'Inbox/R/Commits': Invalid
mailbox name.
> ---
> where cmusieve did
> ---
> fileinto fileinto "Inbox/R/Commits";
> ---
> this new plugin require changing slashes to dots:
> ---
> fileinto fileinto "Inbox.R.Commits";
This doesn't look like something that has changed for the Sieve 
implementation specifically. Did you try dovecot-sieve-1.1 with dovecot 
1.2 to verify? I'll ask Timo about this. What is the configuration of 
your mail store. Did you perhaps migrate from mbox to maildir?
> I think this changes should be described somewhere. May be, it's
> already needed to create wiki page for libsieve at wiki.dovecot.org?
Well, first one is probably a bug. The second one I am not sure about 
yet. I'll get back to you when I do.

PS: please reply to the mailing list as well. Your problems and 
solutions may interest others as well.

Regards,


-- 
Stephan Bosch
stephan at rename-it.nl

Thank you Timo,
it was my mistake.
In 'plugin {}' section line 'sieve = /var/sievescripts/%n/sieve'
worked for both cmusieve and sieve plugins.
Command 'fileinto "INBOX/foo/bar"' with private namespace
configuration you proposed - also worked in both plugins.
I suspected that they have different behavior: cmusieve saved message into
INBOX/foo/bar, while sieve - into foo/bar as in the deliver log I'm pasting
here. But both messages were filed into the same mail folder. Difference in the
logging, I think, is not a big problem.
---
ls -l Maildir/.foo.bar/new 
total 8
-rw------- 1 seriv seriv 513 Oct 30 01:57
1225331850.M790093P1351.sisyphus.office.omniti.com,W=526
-rw------- 1 seriv seriv 518 Oct 30 01:59
1225331966.M468658P1400.sisyphus.office.omniti.com,W=531
tail -n 18 /var/log/deliver/dovecot-deliver.log
---
deliver(seriv): Oct 30 01:57:30 Info: Loading modules from directory:
/usr/lib64/dovecot/modules/lda/
deliver(seriv): Oct 30 01:57:30 Info: Module loaded:
/usr/lib64/dovecot/modules/lda//lib90_cmusieve_plugin.so
deliver(seriv): Oct 30 01:57:30 Info: Namespace: type=private, prefix=INBOX/,
sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes
deliver(seriv): Oct 30 01:57:30 Info: maildir: root exists (/home/seriv/Maildir)
deliver(seriv): Oct 30 01:57:30 Info: maildir++: root=/home/seriv/Maildir,
index=, control=, inbox=/home/seriv/Maildir
deliver(seriv): Oct 30 01:57:30 Info: cmusieve: Using sieve path:
/var/sieve-scripts/seriv.sieve
deliver(seriv): Oct 30 01:57:30 Info: cmusieve: Executing script
/var/sieve-scripts/seriv.sievec
deliver(seriv): Oct 30 01:57:30 Info: Namespace INBOX/: Using permissions from
/home/seriv/Maildir: mode=0644 gid=-1
deliver(seriv): Oct 30 01:57:30 Info: msgid=<20081030015649.B7936140A88 at
sisyphus.office.omniti.com>: saved mail to INBOX/foo/bar
deliver(seriv): Oct 30 01:59:26 Info: Loading modules from directory:
/usr/lib64/dovecot/modules/lda/
deliver(seriv): Oct 30 01:59:26 Info: Module loaded:
/usr/lib64/dovecot/modules/lda//lib90_sieve_plugin.so
deliver(seriv): Oct 30 01:59:26 Info: Namespace: type=private, prefix=INBOX/,
sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes
deliver(seriv): Oct 30 01:59:26 Info: maildir: root exists (/home/seriv/Maildir)
deliver(seriv): Oct 30 01:59:26 Info: maildir++: root=/home/seriv/Maildir,
index=, control=, inbox=/home/seriv/Maildir
deliver(seriv): Oct 30 01:59:26 Info: sieve: using sieve path:
/var/sieve-scripts/seriv.sieve
deliver(seriv): Oct 30 01:59:26 Info: sieve: opening script
/var/sieve-scripts/seriv.sieve
deliver(seriv): Oct 30 01:59:26 Info: sieve: executing compiled script
/var/sieve-scripts/seriv.sieve
deliver(seriv): Oct 30 01:59:26 Info: sieve:
msgid=<20081030015911.576FC140A88 at sisyphus.office.omniti.com>: stored
mail into mailbox 'foo/bar'

-- 
   WBR,
   Sergey Ivanov.

----- "Timo Sirainen" <tss at iki.fi> wrote:
> On Wed, 2008-10-29 at 20:31 +0100, Stephan Bosch wrote:
> > sergey ivanov schreef:
> > > Hi Stephan,
> > > Thank you for quick fix. I've built it (from changeset
> > > 646:747107b816dc) and it looks like working with dovecot-1.2.
> > > I'm doing some testing before put it into Sisyphus repository
> > > (http://en.altlinux.org).
> > > What I saw - it does not recognize lines like:
> > > ---
> > >   sieve = /var/sievescripts/%n/sieve
> 
> Is this inside protocol lda {} or plugin {}? In former it's not
> supposed
> to be expanded (shouldn't have been with cmusieve either), in latter
> it
> should be.
> 
> > > where cmusieve did
> > > ---
> > > fileinto fileinto "Inbox/R/Commits";
> > > ---
> > > this new plugin require changing slashes to dots:
> > > ---
> > > fileinto fileinto "Inbox.R.Commits";
> 
> With a namespace configuration like:
> 
> namespace private { 
>   separator = /  
>   prefix = INBOX/
>   inbox = yes
> }
> 
> And a script:
> 
> require ["fileinto"];
> fileinto "INBOX/foo/bar";
> 
> It seems to work just fine:
> 
> deliver(tss): Oct 30 00:04:36 Info: sieve: msgid=unspecified: stored
> mail into mailbox 'foo.bar'
> 
> Although it probably should use the full untranslated INBOX/foo/bar
> name
> when logging (same with deliver -m INBOX/foo/bar).

> Finally, after little more than a year, I finished the first release of  
> the new Sieve implementation for Dovecot.
Great! I immediately tried to put this into pkgsrc, but ...
> The compilation procedure is identical to the cmusieve plugin  
> (http://wiki.dovecot.org/LDA/Sieve).
I cannot see how this part of configure

if ! test -f "$dovecotdir/dovecot-config"; then
  echo
  echo "dovecot-config not found from $dovecotdir, use
--with-dovecot=PATH"
  echo "to give path to compiled Dovecot sources or to a directory with
the"
  echo "installed dovecot-config file."
  AC_MSG_ERROR([dovecot-config not found])
fi

if test -d "$dovecotdir/src"; then
  # compiling against sources
  have_dovecot_libs=yes
else
  # compiling against installed headers
  echo 
  echo "Cannot compile against the installed headers only."
  AC_MSG_ERROR([dovecot-source not found]);
fi

is supposed to work. Either I point $dovecotdir to the source directory
and get a complaint about a missing dovecot-config or I point it to
the library directory and get a complaint that it can't compile
against headers only.