Hi!
"Dovecot always logs a detailed error message if something goes wrong.
If it doesn't, it's considered a bug and will be fixed." (http://
wiki2.dovecot.org/Logging)
I'm trying to set up a spam filter with dovecot-antispam and dspam as
backend.
When I move a spam message from INBOX to Spam, I get this in syslog:
Nov 28 21:15:58 desktop imap: antispam: mailbox_is_unsure(Spam): 0
Nov 28 21:15:58 desktop imap: antispam: mailbox_is_trash(INBOX): 0
Nov 28 21:15:58 desktop imap: antispam: mailbox_is_trash(Spam): 0
Nov 28 21:15:58 desktop imap: antispam: mail copy: from trash: 0, to trash: 0
Nov 28 21:15:58 desktop imap: antispam: mailbox_is_spam(INBOX): 0
Nov 28 21:15:58 desktop imap: antispam: mailbox_is_spam(Spam): 1
Nov 28 21:15:58 desktop imap: antispam: mailbox_is_unsure(INBOX): 0
Nov 28 21:15:58 desktop imap: antispam: mail copy: src spam: 0, dst spam: 1,
src unsure: 0
There are no log entries about how dspam is called, or what goes on. Indeed,
dspam doesn't get called at all. I know, because I've examined the
antispam-
plugin source code. There would be a log message if dspam was called.
It's hard to study the source code, because there are hardly any comments.
I have been able to trace the problem to function signature_extract_to_list()
in signature.c. There, -1 is returned:
int signature_extract_to_list(const struct signature_config *cfg,
struct mailbox_transaction_context *t,
struct mail *mail, struct siglist **list,
enum classification wanted)
{
const char *const *signatures;
struct siglist *item;
signatures = get_mail_headers(mail, cfg->signature_hdr);
if (!signatures || !signatures[0]) {
if (!cfg->signature_nosig_ignore) {
mail_storage_set_error(t->box->storage,
ME(NOTPOSSIBLE)
"antispam signature not found");
return -1; /* <-- HERE */
} else {
return 0;
}
}
while (signatures[1])
signatures++;
item = i_new(struct siglist, 1);
item->next = *list;
item->wanted = wanted;
item->sig = i_strdup(signatures[0]);
*list = item;
return 0;
}
So, what's happening is "antispam signature not found". My
question: What
does this mean? What's going on?
My "dovecot -n" output is attached.
Bye
Volker
-------------- next part --------------
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
# OS: Linux 4.4.0-47-generic x86_64 Ubuntu 16.04.1 LTS
auth_username_format = %n
auth_verbose = yes
hostname = volker-wysk.de
log_path = /var/log/dovecot.log
mail_debug = yes
mail_location = mdbox:~/lib/Dovecot-Mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
mime foreverypart extracttext
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Entw?rfe {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Papierkorb {
special_use = \Trash
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
mailbox "Versendete Nachrichten" {
special_use = \Sent
}
prefix =
separator = .
}
passdb {
driver = pam
}
plugin {
antispam_backend = dspam
antispam_debug_target = syslog
antispam_dspam_binary = /usr/bin/dspam
antispam_dspam_env = HOME=%h;USER=%u
antispam_signature = X-DSPAM-Signature
antispam_signature_missing = error
antispam_spam = Spam
antispam_trash = Papierkorb
antispam_verbose_debug = 1
sieve = ~/.dovecot.sieve
sieve_dir = ~/lib/Sieve
}
postmaster_address = post at volker-wysk.de
protocols = imap lmtp sieve
ssl = required
ssl_cert = </etc/ssl/certs/dovecot.pem
ssl_key = </etc/ssl/private/dovecot.pem
userdb {
driver = passwd
}
protocol lmtp {
mail_plugins = " sieve mail_log notify"
}
protocol imap {
mail_max_userip_connections = 50
mail_plugins = " antispam"
}
One more point: The spam message, which has been moved to the Spam folder for training dspam, doesn't get moved. When I refresh (in KMail), the message reappears in INBOX. Bye V.W.
Op 28-11-2016 om 21:28 schreef Volker Wysk:> Hi! > > "Dovecot always logs a detailed error message if something goes wrong. > If it doesn't, it's considered a bug and will be fixed." (http:// > wiki2.dovecot.org/Logging)Yes, that is indeed what Dovecot and Pigeonhole do. However, AntiSpam is not part of the normal Dovecot distribution, nor is it a plugin created by Dovecot developers. From what I know, Dovecot has mirrored/hosted the AntiSpam Mercurial repository in the past and contributed a few small fixes, but nothing more. Also, it doesn't look like it is actively maintained at the moment. Pigeonhole Sieve recently gained support for the IMAPSieve extension (http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve). When combined with the exprograms plugin (http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms), you should be able achieve something quite similar using a Sieve script, although I have never tried that myself. Regards, Stephan.
Am Dienstag, 29. November 2016, 13:59:52 CET schrieb Stephan Bosch:> However, AntiSpam is > not part of the normal Dovecot distribution, nor is it a plugin created > by Dovecot developers. From what I know, Dovecot has mirrored/hosted the > AntiSpam Mercurial repository in the past and contributed a few small > fixes, but nothing more. Also, it doesn't look like it is actively > maintained at the moment.So, if I would debug the antispam plugin, adding more log messages, finding the reason it doesn't work now, would my changes be included in the antispam distribution? Do you know any documentation about the Dovecot plugin API? I couldn't find any.> Pigeonhole Sieve recently gained support for the IMAPSieve extension > (http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve). When > combined with the exprograms plugin > (http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms), you > should be able achieve something quite similar using a Sieve script, > although I have never tried that myself.Thanks for the hint. This looks interesing, but also quite complicated. I've read the two pages you specified. Is there any good documentation? I have Peer Heinlein's Dovecot book. But there isn't much in it about plugins. Bye Volker
On 11/28/2016 3:28 PM, Volker Wysk wrote:> Hi! > > "Dovecot always logs a detailed error message if something goes wrong. > If it doesn't, it's considered a bug and will be fixed." (http:// > wiki2.dovecot.org/Logging) > > I'm trying to set up a spam filter with dovecot-antispam and dspam as backend. > > When I move a spam message from INBOX to Spam, I get this in syslog: > > Nov 28 21:15:58 desktop imap: antispam: mailbox_is_unsure(Spam): 0 > Nov 28 21:15:58 desktop imap: antispam: mailbox_is_trash(INBOX): 0 > Nov 28 21:15:58 desktop imap: antispam: mailbox_is_trash(Spam): 0 > Nov 28 21:15:58 desktop imap: antispam: mail copy: from trash: 0, to trash: 0 > Nov 28 21:15:58 desktop imap: antispam: mailbox_is_spam(INBOX): 0 > Nov 28 21:15:58 desktop imap: antispam: mailbox_is_spam(Spam): 1 > Nov 28 21:15:58 desktop imap: antispam: mailbox_is_unsure(INBOX): 0 > Nov 28 21:15:58 desktop imap: antispam: mail copy: src spam: 0, dst spam: 1, > src unsure: 0 > > There are no log entries about how dspam is called, or what goes on. Indeed, > dspam doesn't get called at all. I know, because I've examined the antispam- > plugin source code. There would be a log message if dspam was called. > > It's hard to study the source code, because there are hardly any comments. > > I have been able to trace the problem to function signature_extract_to_list() > in signature.c. There, -1 is returned: > > int signature_extract_to_list(const struct signature_config *cfg, > struct mailbox_transaction_context *t, > struct mail *mail, struct siglist **list, > enum classification wanted) > { > const char *const *signatures; > struct siglist *item; > > signatures = get_mail_headers(mail, cfg->signature_hdr); > if (!signatures || !signatures[0]) { > if (!cfg->signature_nosig_ignore) { > > mail_storage_set_error(t->box->storage, > ME(NOTPOSSIBLE) > "antispam signature not found"); > return -1; /* <-- HERE */ > } else { > return 0; > } > } > > while (signatures[1]) > signatures++; > > item = i_new(struct siglist, 1); > item->next = *list; > item->wanted = wanted; > item->sig = i_strdup(signatures[0]); > > *list = item; > > return 0; > } > > So, what's happening is "antispam signature not found". My question: What > does this mean? What's going on? >I too encountered this same issue after upgrading to 2.2.27. I traced it to an apparent change in Dovecot's mail_get_headers() function, which now returns a non-zero value even when the header is found. This wasn't the behavior in 2.2.24, though I'm not sure if it was in .25 or .26. I haven't tracked down the details further. Anyway, the antispam plugin uses this API to find the X-DSPAM-Signature header, and the change causes the plugin to abort. This happens in the plugin's antispam-plugin.h wrapper, which I modified as follows. I'm confused why the dovecot API behavior changed, but I hope this helps! Tom. *** antispam-plugin.h 2016-12-17 17:25:48.571704442 -0500 --- antispam-plugin.h.orig 2016-12-17 17:24:55.010093416 -0500 *************** *** 268,275 **** static inline const char *const * get_mail_headers(struct mail *mail, const char *hdr) { ! const char *const *ret = NULL; ! mail_get_headers(mail, hdr, &ret); return ret; } --- 268,276 ---- static inline const char *const * get_mail_headers(struct mail *mail, const char *hdr) { ! const char *const *ret; ! if (mail_get_headers(mail, hdr, &ret)) ! return NULL; return ret; }
Am Samstag, 17. Dezember 2016, 17:57:54 CET schrieb Tom Talpey:> I too encountered this same issue after upgrading to 2.2.27. I traced it > to an apparent change in Dovecot's mail_get_headers() function, which > now returns a non-zero value even when the header is found. This wasn't > the behavior in 2.2.24, though I'm not sure if it was in .25 or .26. > I haven't tracked down the details further. > > Anyway, the antispam plugin uses this API to find the X-DSPAM-Signature > header, and the change causes the plugin to abort. This happens in the > plugin's antispam-plugin.h wrapper, which I modified as follows. > > I'm confused why the dovecot API behavior changed, but I hope this > helps!Thank you for your help, but I've already done it differently now. I'm using the spam filter capabilities of my mail reader (KMail). This uses an external spam filter (I've chosen CRM114). That's not optimal, because I have two spam filters now (on my desktop and my laptop), and no spam filter for my smartphone, but it works. Bye Volker
On 18 Dec 2016, at 0.57, Tom Talpey <tom at talpey.com> wrote:> > > I too encountered this same issue after upgrading to 2.2.27. I traced it > to an apparent change in Dovecot's mail_get_headers() function, which > now returns a non-zero value even when the header is found. This wasn't > the behavior in 2.2.24, though I'm not sure if it was in .25 or .26. > I haven't tracked down the details further. > > Anyway, the antispam plugin uses this API to find the X-DSPAM-Signature > header, and the change causes the plugin to abort. This happens in the > plugin's antispam-plugin.h wrapper, which I modified as follows. > > I'm confused why the dovecot API behavior changed, but I hope this > helps!It was mostly an accident. There was a mismatch between mail_get_headers() and mail_get_headers_utf8() return values, which was fixed the wrong way. Also I think there might have been some code that assumed that mail_get_headers() would return > 0 on success. Anyway, too late to change back now...> ! const char *const *ret = NULL; > ! mail_get_headers(mail, hdr, &ret); > return ret;That works, but maybe a nicer way that works for both old and new API and checks also the return value: if (mail_get_headers(mail, hdr, &ret) < 0) return NULL;> } > > --- 268,276 ---- > static inline const char *const * > get_mail_headers(struct mail *mail, const char *hdr) > { > ! const char *const *ret; > ! if (mail_get_headers(mail, hdr, &ret)) > ! return NULL; > return ret; > }