dovecot - May 2010 - sieve spamtest extension

Hello Stephan,

First of all, thanks for time you spent implementing this useful extension.

I have found some ugly behavior with such config:

  sieve_extensions = +notify +spamtest
  sieve_spamtest_status_type = score
  sieve_spamtest_status_header = X-Spam-Scan-Score:
(-?[[:digit:]]+\.[[:digit:]])
  sieve_spamtest_max_value = 10

Sieve:
require
["fileinto","envelope","imap4flags","spamtest","relational","comparator-i;ascii-numeric"];
# rule:[spam]
if anyof (spamtest :value "ge" :comparator "i;ascii-numeric"
"6")
{
	fileinto "Spam";
}

If X-Spam-Scan-Score have 6-9 digit after dot - the first one ignored, thus
X-Spam-Scan-Score: 3.6 -> "ge" skips. #right
X-Spam-Scan-Score: 3.8 -> "ge" works. #wrong
X-Spam-Scan-Score: 5.9 -> "ge" works. #wrong
X-Spam-Scan-Score: 6.0 -> "ge" works. #right
X-Spam-Scan-Score: 6.9 -> "ge" works. #right
X-Spam-Scan-Score: 7.3 -> "ge" works. #right

If the rule modified to (spamtest :value "ge" :comparator
"i;ascii-numeric" "6.0") everything works as expected.

One more thing - sieve-test. I couldn't test my scripts with it:

 # sieve-test -x +spamtest -t sieve spam.txt 
00000007: SPAMTEST test
00000007: spamtest: extension not configured
00000013: JMPFALSE (false)

Performed actions:

  (none)

Implicit keep:

 * store message in folder: INBOX

Info: final result: success

Nikita Koshikov wrote:
> If X-Spam-Scan-Score have 6-9 digit after dot - the first one ignored, thus
> X-Spam-Scan-Score: 3.6 -> "ge" skips. #right
> X-Spam-Scan-Score: 3.8 -> "ge" works. #wrong
> X-Spam-Scan-Score: 5.9 -> "ge" works. #wrong
> X-Spam-Scan-Score: 6.0 -> "ge" works. #right
> X-Spam-Scan-Score: 6.9 -> "ge" works. #right
> X-Spam-Scan-Score: 7.3 -> "ge" works. #right
> 
Note that the value '0' has a special meaning for the spamtest test. The
'definitely not spam' value is '1'. Therefore, values ranging
from 0.0
to 10.0 are calculated into a score as follows:

score = (header_value / 10.0) * 9 + 1

So, the 5.9 match is correct as it yields 6.3, matching against "6".

The 3.8 match, however, is not correct. Please run the attached 
testsuite script I composed from your e-mail as follows:

src/testsuite/testsuite nikita-spamtest.svtest

At my end, this test case succeeds fully, including the 3.8 match. The 
test suite script should be fairly easy to understand and modify. If you 
manage to trigger a bug somehow, please let me now.
> If the rule modified to (spamtest :value "ge" :comparator
"i;ascii-numeric" "6.0") everything works as expected.
> 
Matching against "6.0" will end at the first non-digit character,
which
should normally be equal to matching against "6". Very strange that
this
somehow helps. What version are you using?
> One more thing - sieve-test. I couldn't test my scripts with it:
> 
>  # sieve-test -x +spamtest -t sieve spam.txt 
> 00000007: SPAMTEST test
> 00000007: spamtest: extension not configured
> 00000013: JMPFALSE (false)
> 
You need to set the configuration by setting the appropriate config 
values in the environment (e.g. SIEVE_SPAMTEST_MAX_VALUE) while calling 
sieve-test. (For Dovecot v2.0 the tools also still don't use the Dovecot 
configuration, which needs to be fixed before the first release).

Regards,

Stephan.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: nikita-spamtest.svtest
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20100515/7c3f07aa/attachment-0002.pl>

On Sat, 15 May 2010 18:13:38 +0200
Stephan Bosch wrote:
> Nikita Koshikov wrote:
> 
> > If X-Spam-Scan-Score have 6-9 digit after dot - the first one ignored,
thus
> > X-Spam-Scan-Score: 3.6 -> "ge" skips. #right
> > X-Spam-Scan-Score: 3.8 -> "ge" works. #wrong
> > X-Spam-Scan-Score: 5.9 -> "ge" works. #wrong
> > X-Spam-Scan-Score: 6.0 -> "ge" works. #right
> > X-Spam-Scan-Score: 6.9 -> "ge" works. #right
> > X-Spam-Scan-Score: 7.3 -> "ge" works. #right
> > 
> 
> Note that the value '0' has a special meaning for the spamtest
test. The
> 'definitely not spam' value is '1'. Therefore, values
ranging from 0.0
> to 10.0 are calculated into a score as follows:
> 
> score = (header_value / 10.0) * 9 + 1
> 
> So, the 5.9 match is correct as it yields 6.3, matching against
"6".
> 
> The 3.8 match, however, is not correct. Please run the attached 
> testsuite script I composed from your e-mail as follows:
> 
Today I rerun all test with values from 0.0 to 10.9, everything is working fine
(starting from value 5.6 the rule begun to work). I can't reproduce
yesterday's triggering, either with "6" or "6.0". Seemed
I'm done something wrong.

Sorry for the noise.
> src/testsuite/testsuite nikita-spamtest.svtest
> 
> At my end, this test case succeeds fully, including the 3.8 match. The 
> test suite script should be fairly easy to understand and modify. If you 
> manage to trigger a bug somehow, please let me now.
>
Test case: /root/ive/sieve-spamteset.svtest:

 1: Test 'Value: 3.6' SUCCEEDED
 2: Test 'Value: 3.8' SUCCEEDED
 3: Test 'Value: 5.9' SUCCEEDED
 4: Test 'Value: 6.0' SUCCEEDED
 5: Test 'Value: 6.9' SUCCEEDED
 6: Test 'Value: 7.3' SUCCEEDED

PASS: 6 tests succeeded.
> > If the rule modified to (spamtest :value "ge" :comparator
"i;ascii-numeric" "6.0") everything works as expected.
> > 
> 
> Matching against "6.0" will end at the first non-digit character,
which
> should normally be equal to matching against "6". Very strange
that this
> somehow helps. What version are you using?
> 
> > One more thing - sieve-test. I couldn't test my scripts with it:
> > 
> >  # sieve-test -x +spamtest -t sieve spam.txt 
> > 00000007: SPAMTEST test
> > 00000007: spamtest: extension not configured
> > 00000013: JMPFALSE (false)
> > 
> 
> You need to set the configuration by setting the appropriate config 
> values in the environment (e.g. SIEVE_SPAMTEST_MAX_VALUE) while calling 
> sieve-test. (For Dovecot v2.0 the tools also still don't use the
Dovecot
> configuration, which needs to be fixed before the first release).
Thanks for clarifications.