netfilter buglog - Jan 2018 - [Bug 1221] New: "fib" produces strange results with an IPv6 default route

https://bugzilla.netfilter.org/show_bug.cgi?id=1221

            Bug ID: 1221
           Summary: "fib" produces strange results with an IPv6
default
                    route
           Product: nftables
           Version: unspecified
          Hardware: x86_64
                OS: Debian GNU/Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: kernel
          Assignee: pablo at netfilter.org
          Reporter: f30 at f30.me

I am trying to implement reverse path filtering using "fib" rules like
`fib
saddr . iif oif 0 drop`.
I don't understand why exactly (see #1220), but this generally works for
IPv4
and IPv6 without a default route. However, "fib" starts to behave
strangely
with a v6 default route.

Assume a host with two interfaces, enp0s5 and enp0s6, and the following IP
addresses:

  > ip a
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
group
default qlen 1000
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      inet 127.0.0.1/8 scope host lo
         valid_lft forever preferred_lft forever
      inet6 ::1/128 scope host
         valid_lft forever preferred_lft forever
  2: enp0s5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state
UP group default qlen 1000
      link/ether 00:1c:42:87:bf:33 brd ff:ff:ff:ff:ff:ff
      inet 10.0.0.3/24 brd 10.0.0.255 scope global enp0s5
         valid_lft forever preferred_lft forever
      inet6 fd00::3/64 scope global
         valid_lft forever preferred_lft forever
      inet6 fe80::21c:42ff:fe87:bf33/64 scope link
         valid_lft forever preferred_lft forever
  3: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state
UP group default qlen 1000
      link/ether 00:1c:42:5c:28:c0 brd ff:ff:ff:ff:ff:ff
      inet 10.1.1.3/24 brd 10.1.1.255 scope global enp0s6
         valid_lft forever preferred_lft forever
      inet6 fd11::3/64 scope global
         valid_lft forever preferred_lft forever
      inet6 fe80::21c:42ff:fe5c:28c0/64 scope link
         valid_lft forever preferred_lft forever

The routing table for v6 looks fairly unsurprising:

  > ip -6 r
  fd00::/64 dev enp0s5 proto kernel metric 256 pref medium
  fd11::/64 dev enp0s6 proto kernel metric 256 pref medium
  fe80::/64 dev enp0s5 proto kernel metric 256 pref medium
  fe80::/64 dev enp0s6 proto kernel metric 256 pref medium

Let's add some "fib" rules to get the following ruleset.
(Only using counter statements and adding matches for "enp0s5" and
"enp0s6" for
demonstration purposes. I'm using the "inet" family here, but the
behavior can
also be replicated with "ip6".)

  > nft list ruleset
  table inet filter {
      chain input {
          type filter hook input priority 0; policy accept;
      }

      chain forward {
          type filter hook forward priority 0; policy accept;
      }

      chain output {
          type filter hook output priority 0; policy accept;
      }
  }
  table inet raw {
      chain prerouting {
          type filter hook prerouting priority -300; policy accept;
          fib saddr . iif oif 0 udp dport 1337 counter packets 0 bytes 0
          fib saddr . iif oif "enp0s5" udp dport 1337 counter packets
0 bytes 0
          fib saddr . iif oif "enp0s6" udp dport 1337 counter packets
0 bytes 0
      }
  }

Using Scapy on another host, we now send some packets to the wrong interface
(to enp0s6 with a source address belonging to enp0s5's net):

  >>> e = Ether(dst='00:1c:42:5c:28:c0')
  >>> i = IPv6(src='fd00::1', dst='fd11::3')
  >>> u = UDP(dport=1337)
  >>> sendp(e / i / u, iface='vnic3')
  .
  Sent 1 packets.

In this case, everything works as expected and the first "fib" rule
matches:

  > nft list chain inet raw prerouting
  table inet raw {
      chain prerouting {
          type filter hook prerouting priority -300; policy accept;
          fib saddr . iif oif 0 udp dport 1337 counter packets 1 bytes 48
          fib saddr . iif oif "enp0s5" udp dport 1337 counter packets
0 bytes 0
          fib saddr . iif oif "enp0s6" udp dport 1337 counter packets
0 bytes 0
      }
  }

Let's add a default route now:

  > ip -6 r add default via fd11::1
  > ip -6 r
  fd00::/64 dev enp0s5 proto kernel metric 256 pref medium
  fd11::/64 dev enp0s6 proto kernel metric 256 pref medium
  fe80::/64 dev enp0s5 proto kernel metric 256 pref medium
  fe80::/64 dev enp0s6 proto kernel metric 256 pref medium
  default via fd11::1 dev enp0s6 metric 1024 pref medium

We reset the counters, repeat the Scapy `sendp()` from above and look at the
counters again:

  > nft list chain inet raw prerouting
  table inet raw {
      chain prerouting {
          type filter hook prerouting priority -300; policy accept;
          fib saddr . iif oif 0 udp dport 1337 counter packets 0 bytes 0
          fib saddr . iif oif "enp0s5" udp dport 1337 counter packets
0 bytes 0
          fib saddr . iif oif "enp0s6" udp dport 1337 counter packets
1 bytes
48
      }
  }

By adding a route which should have nothing to do with the respective lookup,
we were able to change the result of our "fib" expressions!
For reference, here's a route lookup for the source address:

  > ip -6 r get fd00::1
  fd00::1 from :: dev enp0s5 proto kernel src fd00::3 metric 256 pref medium

No surprises here, so I suppose the bug is in Netfilter or nftables.
Everything works as expected for IPv4. I observed this with Kernel
4.14.0-3-amd64 and nft 0.8.1 on Debian Testing.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180130/288f3232/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1221

Pablo Neira Ayuso <pablo at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fw at strlen.de

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180201/75397bf1/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1221

Florian Westphal <fw at strlen.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|pablo at netfilter.org         |fw at strlen.de

--- Comment #1 from Florian Westphal <fw at strlen.de> ---
Created attachment 527
  --> https://bugzilla.netfilter.org/attachment.cgi?id=527&action=edit
don

Thanks for the detailed bug report, does this patch fix the bug for you?

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180205/68ff313d/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1221

Felix Dreissig <f30 at f30.me> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |f30 at f30.me

--- Comment #2 from Felix Dreissig <f30 at f30.me> ---
(In reply to Florian Westphal from comment #1)
> Thanks for the detailed bug report, does this patch fix the bug for you?
Thanks for your quick response!
Unfortunately, I can still reproduce the issue even with the patch applied.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180207/84d3ec24/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1221

--- Comment #3 from Florian Westphal <fw at strlen.de> ---
Created attachment 528
  --> https://bugzilla.netfilter.org/attachment.cgi?id=528&action=edit
remove multipath relookup

(In reply to Felix Dreissig from comment #2)
> (In reply to Florian Westphal from comment #1)
> > Thanks for the detailed bug report, does this patch fix the bug for
you?
> 
> Thanks for your quick response!
> Unfortunately, I can still reproduce the issue even with the patch applied.
Indeed, I failed to reproduce your setup correctly.
Attached patch "fixes" the problem, fib inherited this behaviour from
ip6t_rpfilter.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180209/b4a578d6/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1221

--- Comment #4 from Felix Dreissig <f30 at f30.me> ---
(In reply to Florian Westphal from comment #3)
> Attached patch "fixes" the problem, fib inherited this behaviour
from
> ip6t_rpfilter.
Thanks again, I can confirm that this patch fixes the issue.

You're also right about ip6t_rpfilter, I was able to reproduce the buggy
behavior with ip6tables as well o_O. Should that be reported as a separate bug,
or are you gonna handle it as well?

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180209/72815fb1/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1221

--- Comment #5 from Florian Westphal <fw at strlen.de> ---
(In reply to Felix Dreissig from comment #4)
> (In reply to Florian Westphal from comment #3)
> > Attached patch "fixes" the problem, fib inherited this
behaviour from
> > ip6t_rpfilter.
> 
> Thanks again, I can confirm that this patch fixes the issue.
> 
> You're also right about ip6t_rpfilter, I was able to reproduce the
buggy
> behavior with ip6tables as well o_O. Should that be reported as a separate
> bug, or are you gonna handle it as well?
No need, I'll handle it in one go.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180209/9a5fff90/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1221

Florian Westphal <fw at strlen.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED

--- Comment #6 from Florian Westphal <fw at strlen.de> ---
(In reply to Florian Westphal from comment #5)
> > Thanks again, I can confirm that this patch fixes the issue.
> > 
> > You're also right about ip6t_rpfilter, I was able to reproduce the
buggy
> > behavior with ip6tables as well o_O. Should that be reported as a
separate
> > bug, or are you gonna handle it as well?
> 
> No need, I'll handle it in one go.
Patches for both fib and ip6tables rpfilter are now upstream.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180226/bfd87241/attachment.html>