Logcheck devel - Sep 2005 - [patch] performance sucks bricks from my wall

Hiya!

Today, my kernel went berserk and spewed out about 1 gig of debugging logs.

Baaaad kernel.

However, logcheck is equally bad as it has been grinding over this gigabyte
for the last hour and a half, and isn't showing any signs of getting ready
yet. And is causing my machine to not be as responsive as it should be.

This inspired me to write following patchlet to speed up the process
somewhat. Concept-code, completely untested, and of course, doesn't compile.
Might eat your mouse. (This patch is against the version in unstable)

Please let yourselves be inspired by this!

--- /usr/sbin/logcheck	2005-08-22 21:34:29.000000000 +0200
+++ logcheck.new	2005-09-21 14:05:53.055041150 +0200
@@ -316,8 +316,10 @@
 		    debug "Applying Logcheck override files"
 		    for file in $(ls -1 $ignore/) ; do
 		        debug "clean logcheck-<package>: $file"
-			cleanchecked "$ignore/$file"
-		    done
+			cat "$ignore/$file" >>ignores
+                    done
+		    cleanchecked ignores
+#		    rm -f raises
 		else
 		    debug "No Logcheck override files" 
 		fi
@@ -326,8 +328,10 @@
 		# Remove any entries already reported
 		for file in $(ls $raise/ | grep -v '^logcheck') ; do
 		    debug "Cleaning logcheck: $file"
-		    cleanchecked "$raise/$file"
+		    cat "$raise/$file" >> raises
 		done
+                cleanchecked raises
+#                rm -f raises
 	    fi
 
 	    if [ -n "$ignorehigher" ]; then 
@@ -344,8 +348,10 @@
 
 	        # Now apply any local-* files
 	        for file in $(ls -1 $ignore/ | grep '^local-') ; do
-	    	    cleanchecked "$ignore/$file"
+                    cat "$ignore/$file" >>local-ignores
 	        done
+   		cleanchecked local-ignores
+#		rm local-ignores
 	    fi
 
 	    if [ "$(basename $grepfile)" = "logcheck" ]; then


Ciao!

Jasper (who really does like logcheck, but is annoyed by performance issues
like these)
-- 
Jasper Spaans                                       http://jsp.vs19.net/
 14:30:12 up 10444 days,  5:17, 0 users, load average: 4.29 4.67 9.17

               This line was last modified 0 seconds ago.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url :
http://lists.alioth.debian.org/pipermail/logcheck-devel/attachments/20050921/f3568ddf/attachment.pgp

salut,

On Wed, 21 Sep 2005, Jasper Spaans wrote:
> Today, my kernel went berserk and spewed out about 1 gig of debugging logs.
> 
> Baaaad kernel.
urrgs, debian kernel?
 
> However, logcheck is equally bad as it has been grinding over this gigabyte
> for the last hour and a half, and isn't showing any signs of getting
ready
> yet. And is causing my machine to not be as responsive as it should be.
> 
> This inspired me to write following patchlet to speed up the process
> somewhat. Concept-code, completely untested, and of course, doesn't
compile.
> Might eat your mouse. (This patch is against the version in unstable)
> 
> Please let yourselves be inspired by this!
ok thanks for your input,
hadn't enough sleep right now to check the benefits from your idea,
but will do next days.
 
> --- /usr/sbin/logcheck	2005-08-22 21:34:29.000000000 +0200
> +++ logcheck.new	2005-09-21 14:05:53.055041150 +0200
> @@ -316,8 +316,10 @@
>  		    debug "Applying Logcheck override files"
>  		    for file in $(ls -1 $ignore/) ; do
>  		        debug "clean logcheck-<package>: $file"
> -			cleanchecked "$ignore/$file"
> -		    done
> +			cat "$ignore/$file" >>ignores
> +                    done
> +		    cleanchecked ignores
> +#		    rm -f raises
>  		else
>  		    debug "No Logcheck override files" 
>  		fi
> @@ -326,8 +328,10 @@
>  		# Remove any entries already reported
>  		for file in $(ls $raise/ | grep -v '^logcheck') ; do
>  		    debug "Cleaning logcheck: $file"
> -		    cleanchecked "$raise/$file"
> +		    cat "$raise/$file" >> raises
>  		done
> +                cleanchecked raises
> +#                rm -f raises
>  	    fi
>  
>  	    if [ -n "$ignorehigher" ]; then 
> @@ -344,8 +348,10 @@
>  
>  	        # Now apply any local-* files
>  	        for file in $(ls -1 $ignore/ | grep '^local-') ; do
> -	    	    cleanchecked "$ignore/$file"
> +                    cat "$ignore/$file" >>local-ignores
>  	        done
> +   		cleanchecked local-ignores
> +#		rm local-ignores
>  	    fi
>  
>  	    if [ "$(basename $grepfile)" = "logcheck" ]; then
> 
> 
> Ciao!
> 
> Jasper (who really does like logcheck, but is annoyed by performance issues
> like these)
sure that worst case scenario seems not checked yet.

--
maks

Hi Jasper!

On Wed, Sep 21, 2005 at 02:30:28PM +0200, Jasper Spaans
wrote:
> Today, my kernel went berserk and spewed out about 1 gig of debugging logs.
> 
> Baaaad kernel.
Eek!
> However, logcheck is equally bad as it has been grinding over this gigabyte
> for the last hour and a half, and isn't showing any signs of getting
ready
> yet. And is causing my machine to not be as responsive as it should be.
> 
> This inspired me to write following patchlet to speed up the process
> somewhat. Concept-code, completely untested, and of course, doesn't
compile.
> Might eat your mouse. (This patch is against the version in unstable)
> 
> Please let yourselves be inspired by this!
Thanks.  I got this patch working, but I'm not seeing a substantial increase
in performance.  I've attached a corrected version using your idea[0] as a
patch against current CVS.

I'm interested in what others find using this patch.  I have a feeling egrep
is just going to do a linear combination on the regexes whether they are all
in one file or not.

[0] Note that it still does not include disk-full checks.
-- 
Todd Troxell
http://rapidpacket.com/~xtat