Logcheck devel - Jul 2006 - so, about preprocessing... (#376106)

Hi,

I have been given commit access to logcheck by Todd and I am
definitely inclined to help out with rule maintenance, but I would
like to bring #376106 up onto the table.

I've been playing around with my little Makefile and am really
pleased with it. Instead of cryptic regexps, I can just define rules
like so:

  @LEAD@ @PROC_SMTP@: @QUEUE_ID@: @TO@, relay=@DNIP@, @DELAY@,
    @DSNS@, status=deliverable \(@SMTP_SSTATUS@ recipient @EMAIL@
    ok\)@EOL@

which will expand to

  ^[[:upper:]][[:alpha:]]{2} ([[:digit:]]{2}| [[:digit:]])
  ([[:digit:]]{2}:){2}[[:digit:]]{2} seamus
  postfix/smtp\[[[:digit:]]{1,5}\]: (NOQUEUE|[A-F[:digit:]]+):
  to=<([-_.+=[:alnum:]]+@[-_.[:alnum:]]+|[[:alnum:]]+)>(,
  orig_to=<([-_.+=[:alnum:]]+@[-_.[:alnum:]]+|[[:alnum:]]+)>)?,
 
relay=([-_.[:alnum:]]+|([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}|unknown)\[([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}\](:[[:digit:]]{1,5})?,
  delay=[[:digit:]]+(\.[[:digit:]]+)?,
  delays=([[:digit:]]+(\.[[:digit:]]+)?/){3}[[:digit:]]+(\.[[:digit:]]+)?,
  dsn=2\.[[:digit:]]+\.[[:digit:]]+, status=deliverable
  \(2[[:digit:]]{2} recipient
  <([-_.+=[:alnum:]]+@[-_.[:alnum:]]+|[[:alnum:]]+)> ok\)$

OMG you might say, and rightly so... the generated rules are even
less readable to humans, but this way, I can make sure that e.g. an
IP address is always the same:
"([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}" (which could be even more
refined). This makes rule maintenance far easier IMHO, and also
provides for greater consistency in the rules.

I think I could implement this in logcheck non-intrusively, but I'd
want to hear what people have to say first.

So, any comments?

-- 
 .''`.     martin f. krafft <madduck at debian.org>
: :'  :    proud Debian developer and author: http://debiansystem.info
`. `'`
  `-  Debian - when you have better things to do than fixing a system
 
i welcome your constructive criticism and corrections.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature (GPG/PGP)
Url :
http://lists.alioth.debian.org/pipermail/logcheck-devel/attachments/20060704/04938f7a/attachment.pgp

also sprach martin f krafft <madduck at debian.org> [2006.07.04.0052
+0200]:
>   @LEAD@ @PROC_SMTP@: @QUEUE_ID@: @TO@, relay=@DNIP@, @DELAY@,
>     @DSNS@, status=deliverable \(@SMTP_SSTATUS@ recipient @EMAIL@
>     ok\)@EOL@
I just noticed
  http://marc.theaimsgroup.com/?l=logcheck-devel&m=114076370327806&w=2

and the fact that Eric Evans already had this idea a year before me.
I apologise for not having done my research, at least we have two
implementations now (and we both use @VAR@ syntax).

Anyway there's one difference: Eric proposes to compile rules files,
I propose to parse rules at run-time. If I look at
>   ^[[:upper:]][[:alpha:]]{2} ([[:digit:]]{2}| [[:digit:]])
>   ([[:digit:]]{2}:){2}[[:digit:]]{2} seamus
>   postfix/smtp\[[[:digit:]]{1,5}\]: (NOQUEUE|[A-F[:digit:]]+):
>   to=<([-_.+=[:alnum:]]+@[-_.[:alnum:]]+|[[:alnum:]]+)>(,
>   orig_to=<([-_.+=[:alnum:]]+@[-_.[:alnum:]]+|[[:alnum:]]+)>)?,
>  
relay=([-_.[:alnum:]]+|([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}|unknown)\[([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}\](:[[:digit:]]{1,5})?,
>   delay=[[:digit:]]+(\.[[:digit:]]+)?,
>   delays=([[:digit:]]+(\.[[:digit:]]+)?/){3}[[:digit:]]+(\.[[:digit:]]+)?,
>   dsn=2\.[[:digit:]]+\.[[:digit:]]+, status=deliverable
>   \(2[[:digit:]]{2} recipient
>   <([-_.+=[:alnum:]]+@[-_.[:alnum:]]+|[[:alnum:]]+)> ok\)$
then I am moderately sure that a user or even our humble selfs will
prefer to read the more abstract variable-using syntax instead.

I am really in favour of this and would start to implement run-time
translation as soon as I hear people who're also in favour.

-- 
 .''`.     martin f. krafft <madduck at debian.org>
: :'  :    proud Debian developer and author: http://debiansystem.info
`. `'`
  `-  Debian - when you have better things to do than fixing a system
 
"let me take you down, 'cause i'm going to strawberry fields.
 nothing is real and nothing to get hungabout.
 strawberry fields forever."
                                                        -- the beatles
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature (GPG/PGP)
Url :
http://lists.alioth.debian.org/pipermail/logcheck-devel/attachments/20060704/9fdbfdd0/attachment.pgp

On Tue, Jul 04, 2006 at 12:52:35AM +0200, martin f krafft wrote:
[...]
> I think I could implement this in logcheck non-intrusively, but I'd
> want to hear what people have to say first.
> 
> So, any comments?
This is looking pretty good.  The reference code worked for me, and it seems
sane, if a little confusing.  The benefits are pretty obvious, and I'm in
support of the macro idea.  Thus far it's simply lacked a good
implementation.

-- 
Todd Troxell
http://rapidpacket.com/~xtat

martin f krafft wrote:
> Hi,
> 
> I have been given commit access to logcheck by Todd and I am
> definitely inclined to help out with rule maintenance, but I would
> like to bring #376106 up onto the table.
> 
> I've been playing around with my little Makefile and am really
> pleased with it. Instead of cryptic regexps, I can just define rules
> like so:
> 
>   @LEAD@ @PROC_SMTP@: @QUEUE_ID@: @TO@, relay=@DNIP@, @DELAY@,
>     @DSNS@, status=deliverable \(@SMTP_SSTATUS@ recipient @EMAIL@
>     ok\)@EOL@
> 
> which will expand to
> 
>   ^[[:upper:]][[:alpha:]]{2} ([[:digit:]]{2}| [[:digit:]])
>   ([[:digit:]]{2}:){2}[[:digit:]]{2} seamus
>   postfix/smtp\[[[:digit:]]{1,5}\]: (NOQUEUE|[A-F[:digit:]]+):
>   to=<([-_.+=[:alnum:]]+@[-_.[:alnum:]]+|[[:alnum:]]+)>(,
>   orig_to=<([-_.+=[:alnum:]]+@[-_.[:alnum:]]+|[[:alnum:]]+)>)?,
>  
relay=([-_.[:alnum:]]+|([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}|unknown)\[([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}\](:[[:digit:]]{1,5})?,
>   delay=[[:digit:]]+(\.[[:digit:]]+)?,
>   delays=([[:digit:]]+(\.[[:digit:]]+)?/){3}[[:digit:]]+(\.[[:digit:]]+)?,
>   dsn=2\.[[:digit:]]+\.[[:digit:]]+, status=deliverable
>   \(2[[:digit:]]{2} recipient
>   <([-_.+=[:alnum:]]+@[-_.[:alnum:]]+|[[:alnum:]]+)> ok\)$
> 
> OMG you might say, and rightly so... the generated rules are even
> less readable to humans, but this way, I can make sure that e.g. an
> IP address is always the same:
> "([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}" (which could be even
more
> refined). This makes rule maintenance far easier IMHO, and also
> provides for greater consistency in the rules.
> 
> I think I could implement this in logcheck non-intrusively, but I'd
> want to hear what people have to say first.
> 
> So, any comments?
I also like this, not only because of #375428

When looking at the postfix rules, it also crossed my mind that it would
help to use equal regex on equal matches. The Q-ID in postfix's rules is
a good examplex for this. There are various regex that are used here.

bye, Martin

-- 

Powered by Debian GNU / Linux

Browse my blog on http://blog.mein-horde.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
Url :
http://lists.alioth.debian.org/pipermail/logcheck-devel/attachments/20060705/761021fe/attachment.pgp