freebsd stable - Apr 2006 - Fast IPSEC issue

Hello List,

I am running throughput test between to Athlon 64 X2 with Stable 6.x 386 
mode.
I have 100 vpns and 100 gre tunnels. The gres go thru the vpns.
I am using nttcp to do the the thruput testing.

When I use FAST_IPSEC I get better performance than standard ipsec but I 
also get a lot
of the following messages:


Apr 28 10:44:48 J301001 kernel: gre45: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:44:54 J301001 kernel: gre58: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:44:54 J301001 kernel: gre95: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:44:58 J301001 kernel: gre80: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:44:59 J301001 kernel: gre67: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:45:04 J301001 kernel: gre73: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:45:08 J301001 kernel: gre64: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:45:19 J301001 kernel: gre20: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:45:19 J301001 kernel: gre54: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:45:24 J301001 kernel: gre72: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:45:28 J301001 kernel: gre14: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:45:28 J301001 kernel: gre50: gre_output: recursively called 
too many t
imes(2)
Apr 28 10:47:24 J301001 kernel: gre4: gre_output: recursively called too 
many ti
mes(2)

Any ideas why this might be happening with FAST_IPSEC? Should I go back 
to the standard
kame ipsec?

Thanks for your insight,
Steve

-- 

"They that give up essential liberty to obtain temporary safety, 
deserve neither liberty nor safety."  (Ben Franklin)

"The course of history shows that as a government grows, liberty 
decreases."  (Thomas Jefferson)

On Fri, Apr 28, 2006 at 10:57:48AM -0400, Stephen Clark
wrote:
> Apr 28 10:44:48 J301001 kernel: gre45: gre_output: recursively called 
> too many times(2)
This means a packet was fed into if_gre while if_gre was also active.
In the case of FAST_IPSEC this may be normal if you have packets moving
between tunnels. You can change how many levels of recursive call
are permitted by changing the sysctl:

	net.link.gre.max_nesting

Try making it slightly larger and see if the messages stop.

	David.