bugzilla-daemon at netfilter.org
2024-Apr-03 22:14 UTC
[Bug 1742] New: using nfqueue breaks SCTP connection (tracking)
https://bugzilla.netfilter.org/show_bug.cgi?id=1742 Bug ID: 1742 Summary: using nfqueue breaks SCTP connection (tracking) Product: libnetfilter_queue Version: unspecified Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: libnetfilter_queue Assignee: netfilter-buglog at lists.netfilter.org Reporter: antonio.ojea.garcia at gmail.com I'm using a golang library for interacting with nfqueue, it is a very simple logic, I add the following rule table inet kube-netpol { comment "rules for kubernetes NetworkPolicy" chain forward { type filter hook forward priority filter - 5; policy accept; ct state new queue to 100 } } and in userspace I process the packet to emit a verdict. Everything works fine with TCP and UDP, but when using SCTP I can see the packet its modified and breaks the establishment of the connection, more details in https://github.com/aojea/kube-netpol/issues/8 Once I remove the `nfqueue` rule the SCTP connection is established correctly. I triple checked the userspace program accepts the packet and removing the nfqueue rules makes the connection work. I've added a trace (by the way kudos for the tracing functionality is really a great improvement) and I can see how the packet is dropped in a rule that drops connections with invalid state trace id 0329b184 ip filter trace_chain packet: iif "eth0" ether saddr 02:42:c0:a8:08:02 ether daddr 02:42:c0:a8:08:03 ip saddr 10.244.1.47 ip daddr 10.244.2.47 ip dscp cs0 ip ecn ect0 ip ttl 63 ip id 0 ip length 68 sctp sport 47261 sctp dport 8080 sctp vtag 0 @th,96,64 0x10000240486b6e3 trace id 0329b184 ip filter trace_chain rule ip protocol sctp meta nftrace set 1 (verdict continue) trace id 0329b184 ip filter trace_chain verdict continue trace id 0329b184 ip filter trace_chain policy accept trace id 0329b184 inet kube-netpol forward packet: iif "eth0" oif "vetha2b65671" ether saddr 02:42:c0:a8:08:02 ether daddr 02:42:c0:a8:08:03 ip saddr 10.244.1.47 ip daddr 10.244.2.47 ip dscp cs0 ip ecn ect0 ip ttl 62 ip id 0 ip protocol sctp ip length 68 sctp sport 47261 sctp dport 8080 sctp vtag 0 @th,96,64 0x10000240486b6e3 trace id 0329b184 inet kube-netpol forward verdict continue trace id 0329b184 inet kube-netpol forward policy accept trace id 0329b184 ip filter FORWARD packet: iif "eth0" oif "vetha2b65671" ether saddr 02:42:c0:a8:08:02 ether daddr 02:42:c0:a8:08:03 ip saddr 10.244.1.47 ip daddr 10.244.2.47 ip dscp cs0 ip ecn ect0 ip ttl 62 ip id 0 ip length 68 sctp sport 47261 sctp dport 8080 sctp vtag 0 @th,96,64 0x10000240486b6e3 trace id 0329b184 ip filter FORWARD rule counter packets 5735 bytes 2667239 jump KUBE-FORWARD (verdict jump KUBE-FORWARD) trace id 0329b184 ip filter KUBE-FORWARD rule ct state invalid counter packets 8 bytes 544 drop (verdict drop) if I remove the nfqueue rule the packet goes through trace id 058bdf29 ip filter trace_chain packet: iif "eth0" ether saddr 02:42:c0:a8:08:02 ether daddr 02:42:c0:a8:08:03 ip saddr 10.244.1.47 ip daddr 10.244.2.47 ip dscp cs0 ip ecn ect0 ip ttl 63 ip id 0 ip length 68 sctp sport 33348 sctp dport 8080 sctp vtag 0 @th,96,64 0x10000244fde5e72 trace id 058bdf29 ip filter trace_chain rule ip protocol sctp meta nftrace set 1 (verdict continue) trace id 058bdf29 ip filter trace_chain verdict continue trace id 058bdf29 ip filter trace_chain policy accept trace id 058bdf29 ip nat PREROUTING packet: iif "eth0" ether saddr 02:42:c0:a8:08:02 ether daddr 02:42:c0:a8:08:03 ip saddr 10.244.1.47 ip daddr 10.244.2.47 ip dscp cs0 ip ecn ect0 ip ttl 63 ip id 0 ip length 68 sctp sport 33348 sctp dport 8080 sctp vtag 0 @th,96,64 0x10000244fde5e72 trace id 058bdf29 ip nat PREROUTING rule counter packets 17924 bytes 1098260 jump KUBE-SERVICES (verdict jump KUBE-SERVICES) trace id 058bdf29 ip nat KUBE-SERVICES verdict continue trace id 058bdf29 ip nat PREROUTING verdict continue trace id 058bdf29 ip nat PREROUTING policy accept trace id 058bdf29 ip filter FORWARD packet: iif "eth0" oif "vetha2b65671" ether saddr 02:42:c0:a8:08:02 ether daddr 02:42:c0:a8:08:03 ip saddr 10.244.1.47 ip daddr 10.244.2.47 ip dscp cs0 ip ecn ect0 ip ttl 62 ip id 0 ip length 68 sctp sport 33348 sctp dport 8080 sctp vtag 0 @th,96,64 0x10000244fde5e72 trace id 058bdf29 ip filter FORWARD rule ct state new counter packets 2894 bytes 195836 jump KUBE-PROXY-FIREWALL (verdict jump KUBE-PROXY-FIREWALL) trace id 058bdf29 ip filter KUBE-PROXY-FIREWALL verdict continue trace id 058bdf29 ip filter FORWARD rule counter packets 5800 bytes 2671691 jump KUBE-FORWARD (verdict jump KUBE-FORWARD) trace id 058bdf29 ip filter KUBE-FORWARD verdict continue trace id 058bdf29 ip filter FORWARD rule ct state new counter packets 2832 bytes 191716 jump KUBE-SERVICES (verdict jump KUBE-SERVICES) trace id 058bdf29 ip filter KUBE-SERVICES verdict continue trace id 058bdf29 ip filter FORWARD rule ct state new counter packets 2826 bytes 191324 jump KUBE-EXTERNAL-SERVICES (verdict jump KUBE-EXTERNAL-SERVICES) trace id 058bdf29 ip filter KUBE-EXTERNAL-SERVICES verdict continue trace id 058bdf29 ip filter FORWARD verdict continue trace id 058bdf29 ip filter FORWARD policy accept trace id 058bdf29 ip nat POSTROUTING packet: iif "eth0" oif "vetha2b65671" ether saddr 02:42:c0:a8:08:02 ether daddr 02:42:c0:a8:08:03 ip saddr 10.244.1.47 ip daddr 10.244.2.47 ip dscp cs0 ip ecn ect0 ip ttl 62 ip id 0 ip length 68 sctp sport 33348 sctp dport 8080 sctp vtag 0 @th,96,64 0x10000244fde5e72 trace id 058bdf29 ip nat POSTROUTING rule counter packets 5868 bytes 374884 jump KUBE-POSTROUTING (verdict jump KUBE-POSTROUTING) trace id 058bdf29 ip nat KUBE-POSTROUTING verdict return trace id 058bdf29 ip nat POSTROUTING rule fib daddr type != local counter packets 2834 bytes 191394 jump KIND-MASQ-AGENT (verdict jump KIND-MASQ-AGENT) trace id 058bdf29 ip nat KIND-MASQ-AGENT verdict return trace id 058bdf29 ip nat POSTROUTING verdict continue trace id 058bdf29 ip nat POSTROUTING policy accept -- You are receiving this mail because: You are watching all bug changes. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20240403/1ac98ca4/attachment.html>
bugzilla-daemon at netfilter.org
2024-Apr-04 08:42 UTC
[Bug 1742] using nfqueue breaks SCTP connection (tracking)
https://bugzilla.netfilter.org/show_bug.cgi?id=1742 Antonio Ojea <antonio.ojea.garcia at gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 |P2 Component|libnetfilter_queue |netfilter hooks Product|libnetfilter_queue |netfilter/iptables -- You are receiving this mail because: You are watching all bug changes. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20240404/369b6da4/attachment.html>
bugzilla-daemon at netfilter.org
2024-Apr-07 20:27 UTC
[Bug 1742] using nfqueue breaks SCTP connection (tracking)
https://bugzilla.netfilter.org/show_bug.cgi?id=1742 --- Comment #1 from Antonio Ojea <antonio.ojea.garcia at gmail.com> --- I?ve used this great tool from the cilium project (pwru) to trace the packet and I can observe that if I don?t use nfqueue I got this trace 0xffff88810983aa00 0 [<empty>(280243)] ip_forward netns=4026533169 mark=0x0 iface=168(eth0) proto=0x0800 mtu=1500 len=84 192.168.8.2:0->10.244.2.47:0() 0xffff88810983aa00 0 [<empty>(280243)] nf_hook_slow netns=4026533169 mark=0x0 iface=168(eth0) proto=0x0800 mtu=1500 len=84 192.168.8.2:0->10.244.2.47:0() 0xffff88810983aa00 0 [<empty>(280243)] ip_forward_finish netns=4026533169 mark=0x0 iface=168(eth0) proto=0x0800 mtu=1500 len=84 192.168.8.2:0->10.244.2.47:0() and when using it there are some functions that operate on the sctp checksum 0xffff88810749bf00 29 [<empty>(274286)] ip_forward netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 10.244.1.47:0->10.244.2.47:0() 0xffff88810749bf00 29 [<empty>(274286)] nf_hook_slow netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 10.244.1.47:0->10.244.2.47:0() 0xffff88810749bf00 29 [<empty>(274286)] nf_queue netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 10.244.1.47:0->10.244.2.47:0() 0xffff88810749bf00 29 [<empty>(274286)] __nf_queue netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 10.244.1.47:0->10.244.2.47:0() 0xffff88810749bf00 29 [<empty>(274286)] skb_checksum_help netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 10.244.1.47:0->10.244.2.47:0() 0xffff88810749bf00 29 [<empty>(274286)] skb_ensure_writable netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 10.244.1.47:0->10.244.2.47:0() 0xffff88810749bf00 26 [<empty>(3319058)] nf_reroute netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 10.244.1.47:0->10.244.2.47:0() 0xffff88810749bf00 26 [<empty>(3319058)] ip_forward_finish netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 10.244.1.47:0->10.244.2.47:0() The packet captures confirms that when using nfqueue some packet modification happens https://github.com/aojea/kube-netpol/issues/8#issuecomment-2039184720 As a workaround, if I set the flag NFQA_CFG_F_GSO then the packet is not modified and connection works perfectly (I?m only using nfqueue for dropping) -- You are receiving this mail because: You are watching all bug changes. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20240407/08ac02d7/attachment.html>
bugzilla-daemon at netfilter.org
2024-Apr-07 20:34 UTC
[Bug 1742] using nfqueue breaks SCTP connection (tracking)
https://bugzilla.netfilter.org/show_bug.cgi?id=1742 Pablo Neira Ayuso <pablo at netfilter.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |pablo at netfilter.org --- Comment #2 from Pablo Neira Ayuso <pablo at netfilter.org> --- (In reply to Antonio Ojea from comment #1)> > I?ve used this great tool from the cilium project (pwru) to trace the packet > and I can observe that if I don?t use nfqueue I got this trace > > 0xffff88810983aa00 0 [<empty>(280243)] ip_forward > netns=4026533169 mark=0x0 iface=168(eth0) proto=0x0800 mtu=1500 len=84 > 192.168.8.2:0->10.244.2.47:0() > 0xffff88810983aa00 0 [<empty>(280243)] nf_hook_slow > netns=4026533169 mark=0x0 iface=168(eth0) proto=0x0800 mtu=1500 len=84 > 192.168.8.2:0->10.244.2.47:0() > 0xffff88810983aa00 0 [<empty>(280243)] ip_forward_finish > netns=4026533169 mark=0x0 iface=168(eth0) proto=0x0800 mtu=1500 len=84 > 192.168.8.2:0->10.244.2.47:0() > > and when using it there are some functions that operate on the sctp checksum > > 0xffff88810749bf00 29 [<empty>(274286)] ip_forward > netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 > 10.244.1.47:0->10.244.2.47:0() > 0xffff88810749bf00 29 [<empty>(274286)] nf_hook_slow > netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 > 10.244.1.47:0->10.244.2.47:0() > 0xffff88810749bf00 29 [<empty>(274286)] nf_queue > netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 > 10.244.1.47:0->10.244.2.47:0() > 0xffff88810749bf00 29 [<empty>(274286)] __nf_queue > netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 > 10.244.1.47:0->10.244.2.47:0() > 0xffff88810749bf00 29 [<empty>(274286)] skb_checksum_help > netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 > 10.244.1.47:0->10.244.2.47:0() > 0xffff88810749bf00 29 [<empty>(274286)] skb_ensure_writable > netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 > 10.244.1.47:0->10.244.2.47:0() > 0xffff88810749bf00 26 [<empty>(3319058)] nf_reroute > netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 > 10.244.1.47:0->10.244.2.47:0() > 0xffff88810749bf00 26 [<empty>(3319058)] ip_forward_finish > netns=4026532933 mark=0x0 iface=47 proto=0x0800 mtu=1500 len=68 > 10.244.1.47:0->10.244.2.47:0() > > > The packet captures confirms that when using nfqueue some packet > modification happens > https://github.com/aojea/kube-netpol/issues/8#issuecomment-2039184720 > > > As a workaround, if I set the flag NFQA_CFG_F_GSO then the packet is not > modified and connection works perfectly (I?m only using nfqueue for dropping)SCTP checksum is lacking in nfnetlink_queue. There is a patch for nftables that could be used for reference: commit 346e320cb2103edef709c4466a29140c4a8e527a Author: Davide Caratti <dcaratti at redhat.com> Date: Thu Oct 15 18:39:27 2020 +0200 netfilter: nftables: allow re-computing sctp CRC-32C in 'payload' statements There is a skb_checksum_help() call in nfnetlink_queue that needs to be replaced by a helper function that run the specific SCTP checksum function. -- You are receiving this mail because: You are watching all bug changes. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20240407/2b16a70e/attachment-0001.html>
bugzilla-daemon at netfilter.org
2024-Apr-07 20:38 UTC
[Bug 1742] using nfqueue breaks SCTP connection (tracking)
https://bugzilla.netfilter.org/show_bug.cgi?id=1742 --- Comment #3 from Pablo Neira Ayuso <pablo at netfilter.org> --- (In reply to Pablo Neira Ayuso from comment #2)> (In reply to Antonio Ojea from comment #1) > > As a workaround, if I set the flag NFQA_CFG_F_GSO then the packet is not > > modified and connection works perfectly (I?m only using nfqueue for dropping)Workaround makes sense, because branch depends on NFQA_CFG_F_GSO flag: case NFQNL_COPY_PACKET: if (!(queue->flags & NFQA_CFG_F_GSO) && entskb->ip_summed == CHECKSUM_PARTIAL && skb_checksum_help(entskb)) return NULL; if set on, the checksum calculation is skipped. For filtering (only dropping), enabling NFQA_CFG_F_GSO makes perfect sense (unless your heuristic is based on the real packet as seen in the wire, that is, GRO does not interference with the filtering criteria) -- You are receiving this mail because: You are watching all bug changes. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20240407/2005a6a7/attachment.html>
bugzilla-daemon at netfilter.org
2024-Apr-07 20:51 UTC
[Bug 1742] using nfqueue breaks SCTP connection (tracking)
https://bugzilla.netfilter.org/show_bug.cgi?id=1742 Antonio Ojea <antonio.ojea.garcia at gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED --- Comment #4 from Antonio Ojea <antonio.ojea.garcia at gmail.com> ---> There is a skb_checksum_help() call in nfnetlink_queue that needs to be replaced by a helper function that run the specific SCTP checksum function.great, this seems a good opportunity to contribute, let me try to see if I'm able to submit a patch -- You are receiving this mail because: You are watching all bug changes. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20240407/ec79ff63/attachment.html>
Seemingly Similar Threads
- Gluster clients can't see directories that exist or are created within a mounted volume, but can enter them.
- Gluster clients can't see directories that exist or are created within a mounted volume, but can enter them.
- New CentOS Atomic Release and Kubernetes System Containers Now Available
- [Bug 362] Loss of change password functionality
- CentOS-announce Digest, Vol 150, Issue 1