dovecot - Mar 2015 - Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1

Hello Dovecot users,

Last time I had a few stupid problems in the releases, so I'll follow
Timo's example and I release an RC first.

The highlights include the implementation of the index and metadata
extensions. Quite a few bugs are fixed as well.

Changelog v0.4.7:

* editheader extension: Made protection against addition and deletion of
  headers configurable separately. Also, the `Received' and
  `Auto-Submitted' headers are no longer protected against addition by
  default.
* Turned message envelope address parse errors into warnings.
* The interpreter now accepts non-standard domain names, e.g.
  containing '_'.
+ Implemented the Sieve index extension (RFC 5260).
+ Implemented support for the mboxmetadata and servermetadata extensions
  (RFC 5490).
+ Implemented new sieve commands for the doveadm command line utility.
  These commands are currently limited to ManageSieve operations, but
  the other current sieve tools will be migrated to doveadm in the near
  future as well.
+ Added more debug output to binary up-to-date checking.
- The Sieve interpreter now flushes duplicate database during start
  phase of result execution rather than commit phase. This makes sure
  locks on the duplicate database are released as soon as possible,
  preventing contention.
- Performed a few optimizations in the lexical scanner of the language.
- Fixed bug in `:matches' match-type that made a pattern without
  wildcards match as if there were a '*' at the beginning.
- file storage: Restructured storage initialization to address backwards
  compatibility issues.
- Fixed crash in validation of the string parameter of the comparator
  tag.
- extprograms extension: Made sure supplemental group privileges are
  also dropped. This was a problem reported by Debian lintian.
- Fixed bug in handling of binary errors for action side-effects and
  message overrides.

The release is available as follows:

http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc1.tar.gz
http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc1.tar.gz.sig

Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for
more information. Have fun testing this new release and don't hesitate
to notify me when there are any problems.

Regards,

-- 
Stephan Bosch
stephan at rename-it.nl

Hi Stephan,

I'm running the current stable version (Dovecot 2.2.15 and Pigeonhole 
0.4.6), all works fine except an issue.

In my configuration I have this default sieve script (I'm using Maildir):

==require ["vnd.dovecot.duplicate", "fileinto",
"mailbox"];

if header :matches "X-Spam-Status" "Yes, *" {
         fileinto "Spam";
}

if duplicate :seconds 60 {
         fileinto "Trash";
}
==
when in the Maildir/cur/ there are more than (about) 100.000 messages 
dovecot-lda crash and the email is not delivered (remains in the queue) 
and in the log I found this error:

==========Feb 16 03:23:41 mx03 dovecot: lda(user at domain.com): Panic: file 
mail-transaction-log.c: line 271 (mail_transaction_log_rotate): 
assertion failed: (file->locked)
Feb 16 03:23:41 mx03 dovecot: lda(user at domain.com): Error: Raw 
backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x7319a) [0x7f27cef8b19a] 
-> /usr/lib64/dovecot/libdovecot.so.0(i_syslog_fatal_handler+0x33) 
[0x7f27cef8b253] -> /usr/lib64/dovecot/libdovecot.so.0(+0x24b1a) 
[0x7f27cef3cb1a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xd3965) 
[0x7f27cf2ce965] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_write+0x1f5) 
[0x7f27cf2cd7a5] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x2c4) 
[0x7f27cf2c5214] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x510e8) 
[0x7f27cf24c0e8] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(maildir_transaction_save_commit_pre+0x40f)
[0x7f27cf2477ef] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xb39c8) 
[0x7f27cf2ae9c8] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x9f)
[0x7f27cf2bc62f] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(index_transaction_commit+0xa5)
[0x7f27cf2ae585]
-> /usr/lib64/dovecot/lib20_zlib_plugin.so(+0x302f) [0x7f27cd90d02f] -> 
/usr/lib64/dovecot/lib10_quota_plugin.so(+0xac52) [0x7f27cdf34c52] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(+0x96b46) [0x7f27cf291b46] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x4f)
[0x7f27cf279def] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit+0x16) 
[0x7f27cf279ea6] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x418c7) 
[0x7f27cca328c7] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x382a1) 
[0x7f27cca292a1] -> 
/usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x1e2) 
[0x7f27cca295f2] -> 
/usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_execute+0x56) 
[0x7f27cca3a346] -> /usr/lib64/dovecot/lib90_sieve_plugin.so(+0x2e1e) 
[0x7f27ccc8de1e] -> 
/usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver+0x45) 
[0x7f27cf51fda5] -> /usr/libexec/dovecot/dovecot-lda(main+0x6ff) 
[0x402caf] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f27ceba2d5d] 
-> /usr/libexec/dovecot/dovecot-lda() [0x402339]
==========
If I remove from default.sieve "if duplicate" rule all works fine.

Do you think this problem can be solved in the new version?

Thanks

Il 10/03/2015 03:27, Stephan Bosch ha scritto:
> Hello Dovecot users,
>
> Last time I had a few stupid problems in the releases, so I'll follow
> Timo's example and I release an RC first.
>
> The highlights include the implementation of the index and metadata
> extensions. Quite a few bugs are fixed as well.
>
> Changelog v0.4.7:

Alessio Cecchi schreef op 10-3-2015 om 9:26:
>
> ==========> Feb 16 03:23:41 mx03 dovecot: lda(user at domain.com):
Panic: file
> mail-transaction-log.c: line 271 (mail_transaction_log_rotate): 
> assertion failed: (file->locked)
> Feb 16 03:23:41 mx03 dovecot: lda(user at domain.com): Error: Raw 
> backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x7319a) 
> [0x7f27cef8b19a] -> 
> /usr/lib64/dovecot/libdovecot.so.0(i_syslog_fatal_handler+0x33) 
> [0x7f27cef8b253] -> /usr/lib64/dovecot/libdovecot.so.0(+0x24b1a) 
> [0x7f27cef3cb1a] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xd3965) [0x7f27cf2ce965] 
> -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_write+0x1f5) 
> [0x7f27cf2cd7a5] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x2c4) 
> [0x7f27cf2c5214] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x510e8) [0x7f27cf24c0e8] 
> -> 
>
/usr/lib64/dovecot/libdovecot-storage.so.0(maildir_transaction_save_commit_pre+0x40f)
> [0x7f27cf2477ef] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xb39c8) [0x7f27cf2ae9c8] 
> -> 
>
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x9f)
> [0x7f27cf2bc62f] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(index_transaction_commit+0xa5) 
> [0x7f27cf2ae585] -> /usr/lib64/dovecot/lib20_zlib_plugin.so(+0x302f) 
> [0x7f27cd90d02f] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xac52) 
> [0x7f27cdf34c52] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x96b46) [0x7f27cf291b46] 
> -> 
>
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x4f)
> [0x7f27cf279def] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit+0x16)
> [0x7f27cf279ea6] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x418c7) 
> [0x7f27cca328c7] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x382a1) 
> [0x7f27cca292a1] -> 
> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x1e2) 
> [0x7f27cca295f2] -> 
> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_execute+0x56) 
> [0x7f27cca3a346] -> /usr/lib64/dovecot/lib90_sieve_plugin.so(+0x2e1e) 
> [0x7f27ccc8de1e] -> 
> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver+0x45) 
> [0x7f27cf51fda5] -> /usr/libexec/dovecot/dovecot-lda(main+0x6ff) 
> [0x402caf] -> /lib64/libc.so.6(__libc_start_main+0xfd) 
> [0x7f27ceba2d5d] -> /usr/libexec/dovecot/dovecot-lda() [0x402339]
> ==========>
> If I remove from default.sieve "if duplicate" rule all works
fine.
>
> Do you think this problem can be solved in the new version?
Oh, looks like you reported this before. Will discuss this with Timo.

Regards,

Stephan.

On 10 Mar 2015, at 10:26, Alessio Cecchi <alessio at skye.it>
wrote:
> 
> Hi Stephan,
> 
> I'm running the current stable version (Dovecot 2.2.15 and Pigeonhole
0.4.6), all works fine except an issue.
> 
> Feb 16 03:23:41 mx03 dovecot: lda(user at domain.com): Panic: file
mail-transaction-log.c: line 271 (mail_transaction_log_rotate): assertion
failed: (file->locked)
This is fixed by v2.2.16.rc1. I'll make v2.2.16 release this week.
> If I remove from default.sieve "if duplicate" rule all works
fine.
I think it's pure chance that it happens to crash that way. I don't see
how the duplicate checking could have affected this crash.

> Last time I had a few stupid problems in the releases, so
> I'll follow
> Timo's example and I release an RC first.
> 
> The highlights include the implementation of the index and
> metadata
> extensions. Quite a few bugs are fixed as well.
When I compiled and installed this, Sieve scripts were being ignored. Not
sure if it's my own stupid mistake, but when I put v0.4.6 back in place, it
worked fine. No configuration changes, only make install on the different
sources and restart dovecot.

On 3/10/2015 8:11 PM, E.B. wrote:
>> Last time I had a few stupid problems in the releases, so
>> I'll follow
>> Timo's example and I release an RC first.
>>
>> The highlights include the implementation of the index and
>> metadata
>> extensions. Quite a few bugs are fixed as well.
> When I compiled and installed this, Sieve scripts were being ignored. Not
> sure if it's my own stupid mistake, but when I put v0.4.6 back in
place, it
> worked fine. No configuration changes, only make install on the different
> sources and restart dovecot.
Could you show your dovecot -n output?

Also, if you enable mail_debug, what sieve-related debug lines are shown?

Regards,

Stephan.