bugzilla-daemon@bugzilla.netfilter.org
2006-Jul-25 01:09 UTC
[Bug 495] New: Netfilter Connection Tracking Race Condition in Kernel 2.4.x
https://bugzilla.netfilter.org/bugzilla/show_bug.cgi?id=495 Summary: Netfilter Connection Tracking Race Condition in Kernel 2.4.x Product: netfilter/iptables Version: linux-2.4.x Platform: All OS/Version: All Status: NEW Severity: normal Priority: P2 Component: ip_conntrack AssignedTo: laforge@netfilter.org ReportedBy: Bob.Halley@nominum.com Background Our application uses ip_queue in prerouting to divert DNS UDP packets to a userland daemon which inspects them and then issues a NF_ACCEPT or NF_DROP verdict back to the kernel. We found that if several packets with the same conntrack tuple, i.e. the same src addr, src port, dst addr, and dst port, arrive very close together, then only the first one accepted by our software actually makes it back out to the wire; the others are silently dropped. Analysis We instrumented the kernel to find out where the drop was occurring. The code doing the dropping was ip_refrag() in net/ipv4/netfilter/ip_conntrack_standalone.c, specifically: /* We've seen it coming out the other side: confirm */ if (ip_confirm(hooknum, pskb, in, out, okfn) != NF_ACCEPT) return NF_DROP; The dropping is caused by a race between the first packet of a given tuple making it to confirmed state, and the arrival of another packet with the same tuple. If a second packet arrives before the first is confirmed, it is assigned a new connection tracking context instead of joining that of the first unconfirmed packet. When the second packet is finally handled by ip_refrag(), the call to ip_confim() finds that there is already a confirmed entry in the table, and returns NF_DROP.>From the comments in __ip_contrack_confirm(), we infer that this is todeal with duplicated datagrams and some REJECT case, but it's the wrong thing in this case because the subsequent packets are neither duplicates nor REJECTs. We were using RHEL 3 kernel 2.4.21-40 initially. We looked at later 2.4.x kernels and found some promising looking changes, namely the addition of an unconfirmed list, in more recent 2.4.x kernels. We built a 2.4.32 kernel and tested it, but the problem remained. We looked into the nature of the unconfirmed list and discovered that it was solving a different problem, but could be a useful starting point for a fix. Fix We decided to eliminate the race by having subsequent packets with the same conntrack tuple join the conntrack context of the first packet instead of creating a new conntrack context for each of them. Here's the patch: --- linux-2.4.32/net/ipv4/netfilter/ip_conntrack_core.c.orig 2005-04-03 18:42:20.000000000 -0700 +++ linux-2.4.32/net/ipv4/netfilter/ip_conntrack_core.c 2006-07-24 13:23:25.000000000 -0700 @@ -777,6 +777,14 @@ /* look for tuple match */ h = ip_conntrack_find_get(&tuple, NULL); if (!h) { + READ_LOCK(&ip_conntrack_lock); + h = LIST_FIND(&unconfirmed, conntrack_tuple_cmp, + struct ip_conntrack_tuple_hash *, &tuple, NULL); + if (h) + atomic_inc(&h->ctrack->ct_general.use); + READ_UNLOCK(&ip_conntrack_lock); + } + if (!h) { h = init_conntrack(&tuple, proto, skb); if (!h) return NULL; This patch reliably ends the race, and we no longer have mysteriously disappearing packets. Not being netfilter experts, we're not certain that this patch has no other side effects, and would appreciate any advice or alternative fixes that people who know more than we do have to offer. Regards, Bob Halley <Bob.Halley@nominum.com> Brian Wellington <Brian.Wellington@nominum.com> -- Configure bugmail: https://bugzilla.netfilter.org/bugzilla/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
Seemingly Similar Threads
- [SECURITY] Netfilter Security Advisory: Conntrack list_del() DoS
- Re: ip_conntrack_lock not readlocked (fwd)
- [ANNOUNCE] Bug in kernel == 2.4.10 causing netfilter problem
- [Bridge] [PATCH/RFC] Reduce call chain length in netfilter (take 2)
- Problems with Krb5/GSSAPI patches in FBSD 4.3