netflow tools - Jan 2006 - Number of active flows raises and raises...

Hello,

I want to use softflowd for traffic accounting of a mirrored port. The
amount of traffic is currently about 10,000 packets per second.

I use the following parameters:

---8<-----------------------------------------------------------------
/usr/local/sbin/softflowd -i eth2 -t maxlife=300 -m 1048576 -n
127.0.0.1:9800
---8<-----------------------------------------------------------------

I did expect that the number of active flows won''t raise mentionably
after the limit of 5 minutes because softflowd has to expire the flows
(see option "-t maxlife=300"). But the number of active flows raises
and
raises until the limit of 1048576 is reached. I have no answer for that.
Once the limit of maximum flows is reached softflowd takes 99% of the
CPU.

Any suggestions?

Regards,
Andreas

On Mon, 23 Jan 2006, Andreas Brillisauer -- Hetzner Online AG wrote:
> I did expect that the number of active flows won''t raise
mentionably
> after the limit of 5 minutes because softflowd has to expire the flows
> (see option "-t maxlife=300"). But the number of active flows
raises and
> raises until the limit of 1048576 is reached. I have no answer for that.
> Once the limit of maximum flows is reached softflowd takes 99% of the
That is a bug, which caused the maxlife to only be checked when traffic was
received on a flow. Please try this patch:

Index: softflowd.c
==================================================================RCS file:
/var/cvs/softflowd/softflowd.c,v
retrieving revision 1.86
diff -u -p -r1.86 softflowd.c
--- softflowd.c	18 Nov 2005 05:19:12 -0000	1.86
+++ softflowd.c	24 Jan 2006 00:45:01 -0000
@@ -473,7 +473,7 @@ flow_update_expiry(struct FLOWTRACK *ft,
 	if (ft->icmp_timeout != 0 &&
 	    ((flow->af == AF_INET && flow->protocol == IPPROTO_ICMP) || 
 	    ((flow->af == AF_INET6 && flow->protocol ==
IPPROTO_ICMPV6)))) {
-		/* UDP flows */
+		/* ICMP flows */
 		flow->expiry->expires_at = flow->flow_last.tv_sec + 
 		    ft->icmp_timeout;
 		flow->expiry->reason = R_ICMP;
@@ -486,6 +486,11 @@ flow_update_expiry(struct FLOWTRACK *ft,
 	flow->expiry->reason = R_GENERAL;
 
  out:
+	if (ft->maximum_lifetime != 0 && flow->expiry->expires_at !=
0) {
+		flow->expiry->expires_at = MIN(flow->expiry->expires_at,
+		    flow->flow_start.tv_sec + ft->maximum_lifetime);
+	}
+
 	EXPIRY_INSERT(EXPIRIES, &ft->expiries, flow->expiry);
 }
 
@@ -745,9 +750,18 @@ check_expired(struct FLOWTRACK *ft, stru
 		    (ex != CE_EXPIRE_FORCED &&
 		    (expiry->expires_at < now.tv_sec))) {
 			/* Flow has expired */
+
+			if (ft->maximum_lifetime != 0 && 
+	    		    expiry->flow->flow_last.tv_sec - 
+			    expiry->flow->flow_start.tv_sec >= 
+	    		    ft->maximum_lifetime)
+					expiry->reason = R_MAXLIFE;
+
 			if (verbose_flag)
-				logit(LOG_DEBUG, "Queuing flow seq:%llu (%p) for expiry",
-				   expiry->flow->flow_seq, expiry->flow);
+				logit(LOG_DEBUG,
+				    "Queuing flow seq:%llu (%p) for expiry "
+				    "reason %d", expiry->flow->flow_seq,
+				    expiry->flow, expiry->reason);
 
 			/* Add to array of expired flows */
 			oldexp = expired_flows;