On 13-04-16 08:27, Johannes Rohr wrote:> Hi List, > > does the antispam plugin any way of actually seeing how it is > calling the pipe program and how it is failing? I can't reproduce the > failure when I call sa-learn manually. Can someone tell me, with > which UID the call happens, is it the UID the current dovecot process > is running under (which would be vmail, as it is a virtual user > setup) Cheers, >Hi Johannes, This is exactly what the wrapper script is supposed to log for you, or which is trivially to add to its output. The wrapper script is not actually useful for production, but it is for debugging. That is why I asked for its logging, and any differences between manual run and a dovecot run. Regards, Tom> Johannes > > Am 12.04.2016 um 23:09 schrieb Johannes Rohr: >> Hi Tom, >> >> Am 12.04.2016 um 22:26 schrieb Tom Hendrikx: >>> On 12-04-16 20:14, Johannes Rohr wrote: >>>> Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise >>>> with the antispam plugin in version 2.0+20120225-2 and >>>> spamassassin at version 3.2.2 >>>> >>>> I have been trying and failed to get the pipe backend of the >>>> antispam plugin to work. Spamassin by itself works, a manual >>>> call of sa-learn works fine. Bayes data is stored in a mysql >>>> DB. >>>> >>>> I have the following configuration in >>>> /etc/dovecot/conf.d/90-plugin.conf >>>> >>>> plugin { #setting_name = value sieve=~/.dovecot.sieve >>>> sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam >>>> antispam_pipe_program_notspam_arg = --ham >>>> antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh >>>> antispam_pipe_program_args = --username=%u # % expansion done >>>> by dovecot >>> You need to specify the argument list as a list seperated by >>> semicolons, per example on the wiki: >>> >>> antispam_pipe_program_args = --username;%u >> >> Not sure about that, because sa-learn expects the parameter >> --username=user at na.me, this is one parameter, not two, and it >> works for me locally, where I also set up dovecot to test. The >> main differences are that the versions I have here are newer, I'm >> running Debian sid, the server is on Ubuntu Precise, and second, >> the server is handling virtual users, locally I have only real Unix >> users. >> >> Also, dovecot doesn't complain about configuration errors regard >> --username=%u. What other way would be there to get the parameter >> passed to sa-learn the way I want? >> >> btw, I tried this, just to be sure, but the result was the same. >> >>> >>> The verbose debug should give you some logging to work with. >> Yes, and I pasted the output. I can of course repeat: >> >> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr >> 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBOX): 0 Apr 12 >> 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12 >> 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: >> 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 >> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr >> 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12 >> 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1, >> src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running >> mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 >> imap: antispam: running mailtrain backend program >> /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running >> mailtrain backend program parameter 1 --username=johannes at rohr.org >> Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend >> program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run >> program failed with exit code -1 >> >> >> >> >>> Also, what does the wrapper script log when dovecot runs it? >> >> I have actually ditched the wrapper script because it works >> without locally, I have configured the plugin to call sa-learn >> directly. >> >>> How does that differ from a manual run? >> sa-learn works just fine with a manual run. It seems as if the >> plugin is passing the parameters wrongly, but the debugging output >> doesn't reveal the actual command, so it's just guesswork. >> >> Cheers, >> >> Johannes >> >>> >>> >>>> antispam_trash = trash;Trash;Deleted Items;Deleted Messages >>>> antispam_spam = SPAM;Junk antispam_backend = pipe >>>> antispam_verbose_debug = 1 antispam_debug_target = syslog >>>> antispam_pipe_tmpdir = /tmp } >>>> >>>> I have tried the following wrapper script: #!/bin/bash # set >>>> -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> >>>> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> >>>> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D >>>> /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f >>>> /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0 >>>> >>>> It somehow isn't working. I can call the wrapper script >>>> manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh >>>> --username=some at mail.user --ham \ < someham >>>> >>>> But it doesn't work from dovecot. It seems the script is >>>> called. but no new patterns show up in the bayes DB. >>>> Unfortunately, the plugin seems to be documented poorly. How >>>> does it call the programme it pipes to? Is it something like >>>> $programme --par1 --par2 < message ? Why can't / shouldn't I >>>> call sa-learn directly? All the instructions I found on the in >>>> internet contained a wrapper script, but none bothered to >>>> explain why you would actually need it. >>>> >>>> I would be very grateful, if someone could enlighten me on >>>> that... >>>> >>>> Cheers, >>>> >>>> Johannes >>>
Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:> On 13-04-16 08:27, Johannes Rohr wrote: >> Hi List, >> >> does the antispam plugin any way of actually seeing how it is >> calling the pipe program and how it is failing? I can't reproduce the >> failure when I call sa-learn manually. Can someone tell me, with >> which UID the call happens, is it the UID the current dovecot process >> is running under (which would be vmail, as it is a virtual user >> setup) Cheers, >> > Hi Johannes, > http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting > This is exactly what the wrapper script is supposed to log for you, or > which is trivially to add to its output. The wrapper script is not > actually useful for production, but it is for debugging. That is why I > asked for its logging, and any differences between manual run and a > dovecot run.When I used the wrapper script recommended at http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID and parameters (--username=some at na.me --ham). The wrapper script noted in the dovecot wiki seems broken. Note the commented out line: #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt It seems like the second half of the line "cat<&0 >> /tmp/sendmail-msg-$$.txt" was originally a separate line, catting the message, which has been passed by the plugin via STDOUT to a file. Without that command, the call of sa-learn in the next line has no file to read from: /usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f /tmp/sendmail-msg-$$.txt & So I moved the cat command to a separate uncommented line and to be sure I also commented out the rm -f /tmp/sendmail-msg-$$.txt & to verify that the file actually contains something, (why is there an "&" terminating the line, which would have the rm command executed in the background?) But it seems the /tmp/sendmail-msg-$$.txt & was empty and the whole operation continued to fail. Could it be that the plugin fails to pass the actual message? Cheers, Johannes> > Regards, > Tom > > >> Johannes >> >> Am 12.04.2016 um 23:09 schrieb Johannes Rohr: >>> Hi Tom, >>> >>> Am 12.04.2016 um 22:26 schrieb Tom Hendrikx: >>>> On 12-04-16 20:14, Johannes Rohr wrote: >>>>> Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise >>>>> with the antispam plugin in version 2.0+20120225-2 and >>>>> spamassassin at version 3.2.2 >>>>> >>>>> I have been trying and failed to get the pipe backend of the >>>>> antispam plugin to work. Spamassin by itself works, a manual >>>>> call of sa-learn works fine. Bayes data is stored in a mysql >>>>> DB. >>>>> >>>>> I have the following configuration in >>>>> /etc/dovecot/conf.d/90-plugin.conf >>>>> >>>>> plugin { #setting_name = value sieve=~/.dovecot.sieve >>>>> sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam >>>>> antispam_pipe_program_notspam_arg = --ham >>>>> antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh >>>>> antispam_pipe_program_args = --username=%u # % expansion done >>>>> by dovecot >>>> You need to specify the argument list as a list seperated by >>>> semicolons, per example on the wiki: >>>> >>>> antispam_pipe_program_args = --username;%u >>> Not sure about that, because sa-learn expects the parameter >>> --username=user at na.me, this is one parameter, not two, and it >>> works for me locally, where I also set up dovecot to test. The >>> main differences are that the versions I have here are newer, I'm >>> running Debian sid, the server is on Ubuntu Precise, and second, >>> the server is handling virtual users, locally I have only real Unix >>> users. >>> >>> Also, dovecot doesn't complain about configuration errors regard >>> --username=%u. What other way would be there to get the parameter >>> passed to sa-learn the way I want? >>> >>> btw, I tried this, just to be sure, but the result was the same. >>> >>>> The verbose debug should give you some logging to work with. >>> Yes, and I pasted the output. I can of course repeat: >>> >>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr >>> 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): 0 Apr 12 >>> 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12 >>> 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: >>> 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 >>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr >>> 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12 >>> 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1, >>> src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running >>> mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 >>> imap: antispam: running mailtrain backend program >>> /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running >>> mailtrain backend program parameter 1 --username=johannes at rohr.org >>> Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend >>> program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run >>> program failed with exit code -1 >>> >>> >>> >>> >>>> Also, what does the wrapper script log when dovecot runs it? >>> I have actually ditched the wrapper script because it works >>> without locally, I have configured the plugin to call sa-learn >>> directly. >>> >>>> How does that differ from a manual run? >>> sa-learn works just fine with a manual run. It seems as if the >>> plugin is passing the parameters wrongly, but the debugging output >>> doesn't reveal the actual command, so it's just guesswork. >>> >>> Cheers, >>> >>> Johannes >>> >>>> >>>>> antispam_trash = trash;Trash;Deleted Items;Deleted Messages >>>>> antispam_spam = SPAM;Junk antispam_backend = pipe >>>>> antispam_verbose_debug = 1 antispam_debug_target = syslog >>>>> antispam_pipe_tmpdir = /tmp } >>>>> >>>>> I have tried the following wrapper script: #!/bin/bash # set >>>>> -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> >>>>> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> >>>>> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D >>>>> /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f >>>>> /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0 >>>>> >>>>> It somehow isn't working. I can call the wrapper script >>>>> manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh >>>>> --username=some at mail.user --ham \ < someham >>>>> >>>>> But it doesn't work from dovecot. It seems the script is >>>>> called. but no new patterns show up in the bayes DB. >>>>> Unfortunately, the plugin seems to be documented poorly. How >>>>> does it call the programme it pipes to? Is it something like >>>>> $programme --par1 --par2 < message ? Why can't / shouldn't I >>>>> call sa-learn directly? All the instructions I found on the in >>>>> internet contained a wrapper script, but none bothered to >>>>> explain why you would actually need it. >>>>> >>>>> I would be very grateful, if someone could enlighten me on >>>>> that... >>>>> >>>>> Cheers, >>>>> >>>>> Johannes
tweaked the wrapper script to make it work. It seems like the sa-learn call silently fails: "/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben root at vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log starting log 25200-start (--username=johannes at rohr.org --spam) libgcc_s.so.1 must be installed for pthread_cancel to work 25200-end Does the above error message have anything to do with it? Running the script & sa-learn manually works fine. do virtual users change something how dovecot calls programs, such as chrooting them? Cheers, Johannes Am 13.04.2016 um 09:45 schrieb Johannes Rohr:> Am 13.04.2016 um 09:17 schrieb Tom Hendrikx: >> On 13-04-16 08:27, Johannes Rohr wrote: >>> Hi List, >>> >>> does the antispam plugin any way of actually seeing how it is >>> calling the pipe program and how it is failing? I can't reproduce the >>> failure when I call sa-learn manually. Can someone tell me, with >>> which UID the call happens, is it the UID the current dovecot process >>> is running under (which would be vmail, as it is a virtual user >>> setup) Cheers, >>> >> Hi Johannes, >> http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting >> This is exactly what the wrapper script is supposed to log for you, or >> which is trivially to add to its output. The wrapper script is not >> actually useful for production, but it is for debugging. That is why I >> asked for its logging, and any differences between manual run and a >> dovecot run. > > When I used the wrapper script recommended at > http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID > and parameters (--username=some at na.me --ham). The wrapper script noted > in the dovecot wiki seems broken. Note the commented out line: > > #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt > > It seems like the second half of the line "cat<&0 >> > /tmp/sendmail-msg-$$.txt" was originally a separate line, catting the > message, which has been passed by the plugin via STDOUT to a file. > Without that command, the call of sa-learn in the next line has no > file to read from: > > /usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f > /tmp/sendmail-msg-$$.txt & > > So I moved the cat command to a separate uncommented line and to be > sure I also commented out the > > rm -f /tmp/sendmail-msg-$$.txt & > > to verify that the file actually contains something, (why is there an > "&" terminating the line, which would have the rm command executed in > the background?) > > But it seems the > > /tmp/sendmail-msg-$$.txt & > > was empty and the whole operation continued to fail. Could it be that > the plugin fails to pass the actual message? > > Cheers, > > Johannes > >> >> Regards, >> Tom >> >> >>> Johannes >>> >>> Am 12.04.2016 um 23:09 schrieb Johannes Rohr: >>>> Hi Tom, >>>> >>>> Am 12.04.2016 um 22:26 schrieb Tom Hendrikx: >>>>> On 12-04-16 20:14, Johannes Rohr wrote: >>>>>> Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise >>>>>> with the antispam plugin in version 2.0+20120225-2 and >>>>>> spamassassin at version 3.2.2 >>>>>> >>>>>> I have been trying and failed to get the pipe backend of the >>>>>> antispam plugin to work. Spamassin by itself works, a manual >>>>>> call of sa-learn works fine. Bayes data is stored in a mysql >>>>>> DB. >>>>>> >>>>>> I have the following configuration in >>>>>> /etc/dovecot/conf.d/90-plugin.conf >>>>>> >>>>>> plugin { #setting_name = value sieve=~/.dovecot.sieve >>>>>> sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam >>>>>> antispam_pipe_program_notspam_arg = --ham >>>>>> antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh >>>>>> antispam_pipe_program_args = --username=%u # % expansion done >>>>>> by dovecot >>>>> You need to specify the argument list as a list seperated by >>>>> semicolons, per example on the wiki: >>>>> >>>>> antispam_pipe_program_args = --username;%u >>>> Not sure about that, because sa-learn expects the parameter >>>> --username=user at na.me, this is one parameter, not two, and it >>>> works for me locally, where I also set up dovecot to test. The >>>> main differences are that the versions I have here are newer, I'm >>>> running Debian sid, the server is on Ubuntu Precise, and second, >>>> the server is handling virtual users, locally I have only real Unix >>>> users. >>>> >>>> Also, dovecot doesn't complain about configuration errors regard >>>> --username=%u. What other way would be there to get the parameter >>>> passed to sa-learn the way I want? >>>> >>>> btw, I tried this, just to be sure, but the result was the same. >>>> >>>>> The verbose debug should give you some logging to work with. >>>> Yes, and I pasted the output. I can of course repeat: >>>> >>>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr >>>> 12 22:53:02 vm10 imap: antispam: >>>> mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): >>>> 0 Apr 12 >>>> 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12 >>>> 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: >>>> 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 >>>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr >>>> 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12 >>>> 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1, >>>> src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running >>>> mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 >>>> imap: antispam: running mailtrain backend program >>>> /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running >>>> mailtrain backend program parameter 1 --username=johannes at rohr.org >>>> Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend >>>> program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run >>>> program failed with exit code -1 >>>> >>>> >>>> >>>> >>>>> Also, what does the wrapper script log when dovecot runs it? >>>> I have actually ditched the wrapper script because it works >>>> without locally, I have configured the plugin to call sa-learn >>>> directly. >>>> >>>>> How does that differ from a manual run? >>>> sa-learn works just fine with a manual run. It seems as if the >>>> plugin is passing the parameters wrongly, but the debugging output >>>> doesn't reveal the actual command, so it's just guesswork. >>>> >>>> Cheers, >>>> >>>> Johannes >>>> >>>>> >>>>>> antispam_trash = trash;Trash;Deleted Items;Deleted Messages >>>>>> antispam_spam = SPAM;Junk antispam_backend = pipe >>>>>> antispam_verbose_debug = 1 antispam_debug_target = syslog >>>>>> antispam_pipe_tmpdir = /tmp } >>>>>> >>>>>> I have tried the following wrapper script: #!/bin/bash # set >>>>>> -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> >>>>>> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> >>>>>> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D >>>>>> /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f >>>>>> /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0 >>>>>> >>>>>> It somehow isn't working. I can call the wrapper script >>>>>> manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh >>>>>> --username=some at mail.user --ham \ < someham >>>>>> >>>>>> But it doesn't work from dovecot. It seems the script is >>>>>> called. but no new patterns show up in the bayes DB. >>>>>> Unfortunately, the plugin seems to be documented poorly. How >>>>>> does it call the programme it pipes to? Is it something like >>>>>> $programme --par1 --par2 < message ? Why can't / shouldn't I >>>>>> call sa-learn directly? All the instructions I found on the in >>>>>> internet contained a wrapper script, but none bothered to >>>>>> explain why you would actually need it. >>>>>> >>>>>> I would be very grateful, if someone could enlighten me on >>>>>> that... >>>>>> >>>>>> Cheers, >>>>>> >>>>>> Johannes