Linux Ethernet Bridging - Mar 2018 - [Bridge] Problem with bridge (mcast-to-ucast + hairpin) and Broadcom's 802.11f in their FullMAC fw

I've problem when using OpenWrt/LEDE on a home router with Broadcom's
FullMAC WiFi chipset.


First of all OpenWrt/LEDE uses bridge interface for LAN network with:
1) IFLA_BRPORT_MCAST_TO_UCAST
2) Clients isolation in hostapd
3) Hairpin mode enabled

For more details please see Linus's patch description:
https://patchwork.kernel.org/patch/9530669/
and maybe hairpin mode patch:
https://lwn.net/Articles/347344/

Short version: in that setup packets received from a bridged wireless
interface can be handled back to it for transmission.


Now, Broadcom's firmware for their FullMAC chipsets in AP mode
supports an obsoleted 802.11f AKA IAPP standard. It's a roaming
standard that was replaced by 802.11r.

Whenever a new station associates, firmware generates a packet like:
ff ff ff ff  ff ff ec 10  7b 5f ?? ??  00 06 00 01  af 81 01 00
(just masked 2 bytes of my MAC)

For mode details you can see discussion in my brcmfmac patch thread:
https://patchwork.kernel.org/patch/10191451/


The problem is that bridge (in setup as above) handles such a packet
back to the device.

That makes Broadcom's FullMAC firmware believe that a given station
just connected to another AP in a network (which doesn't even exist).
As a result firmware immediately disassociates that station. It's
simply impossible to connect to the router. Every association is
followed by immediate disassociation.


Can you see any solution for this problem? Is that an option to stop
multicast-to-unicast from touching 802.11f packets? Some other ideas?
Obviously I can't modify Broadcom's firmware and drop that obsoleted
standard.

-- 
Rafa?

Sending with a fixed linux-wireless ML address. Please kindly send your
replies using linux-wireless@

On 02/27/2018 11:08 AM, Rafa? Mi?ecki wrote:
> I've problem when using OpenWrt/LEDE on a home router with
Broadcom's
> FullMAC WiFi chipset.
>
>
> First of all OpenWrt/LEDE uses bridge interface for LAN network with:
> 1) IFLA_BRPORT_MCAST_TO_UCAST
> 2) Clients isolation in hostapd
> 3) Hairpin mode enabled
>
> For more details please see Linus's patch description:
> https://patchwork.kernel.org/patch/9530669/
> and maybe hairpin mode patch:
> https://lwn.net/Articles/347344/
>
> Short version: in that setup packets received from a bridged wireless
> interface can be handled back to it for transmission.
>
>
> Now, Broadcom's firmware for their FullMAC chipsets in AP mode
> supports an obsoleted 802.11f AKA IAPP standard. It's a roaming
> standard that was replaced by 802.11r.
>
> Whenever a new station associates, firmware generates a packet like:
> ff ff ff ff  ff ff ec 10  7b 5f ?? ??  00 06 00 01  af 81 01 00
> (just masked 2 bytes of my MAC)
>
> For mode details you can see discussion in my brcmfmac patch thread:
> https://patchwork.kernel.org/patch/10191451/
>
>
> The problem is that bridge (in setup as above) handles such a packet
> back to the device.
>
> That makes Broadcom's FullMAC firmware believe that a given station
> just connected to another AP in a network (which doesn't even exist).
> As a result firmware immediately disassociates that station. It's
> simply impossible to connect to the router. Every association is
> followed by immediate disassociation.
>
>
> Can you see any solution for this problem? Is that an option to stop
> multicast-to-unicast from touching 802.11f packets? Some other ideas?
> Obviously I can't modify Broadcom's firmware and drop that
obsoleted
> standard.

On Tue, 27 Feb 2018 11:08:20 +0100
Rafa? Mi?ecki <zajec5 at gmail.com> wrote:
> I've problem when using OpenWrt/LEDE on a home router with
Broadcom's
> FullMAC WiFi chipset.
> 
> 
> First of all OpenWrt/LEDE uses bridge interface for LAN network with:
> 1) IFLA_BRPORT_MCAST_TO_UCAST
> 2) Clients isolation in hostapd
> 3) Hairpin mode enabled
> 
> For more details please see Linus's patch description:
> https://patchwork.kernel.org/patch/9530669/
> and maybe hairpin mode patch:
> https://lwn.net/Articles/347344/
> 
> Short version: in that setup packets received from a bridged wireless
> interface can be handled back to it for transmission.
> 
> 
> Now, Broadcom's firmware for their FullMAC chipsets in AP mode
> supports an obsoleted 802.11f AKA IAPP standard. It's a roaming
> standard that was replaced by 802.11r.
> 
> Whenever a new station associates, firmware generates a packet like:
> ff ff ff ff  ff ff ec 10  7b 5f ?? ??  00 06 00 01  af 81 01 00
> (just masked 2 bytes of my MAC)
> 
> For mode details you can see discussion in my brcmfmac patch thread:
> https://patchwork.kernel.org/patch/10191451/
> 
> 
> The problem is that bridge (in setup as above) handles such a packet
> back to the device.
> 
> That makes Broadcom's FullMAC firmware believe that a given station
> just connected to another AP in a network (which doesn't even exist).
> As a result firmware immediately disassociates that station. It's
> simply impossible to connect to the router. Every association is
> followed by immediate disassociation.
> 
> 
> Can you see any solution for this problem? Is that an option to stop
> multicast-to-unicast from touching 802.11f packets? Some other ideas?
> Obviously I can't modify Broadcom's firmware and drop that
obsoleted
> standard.
> 
ebtables is your friend in dealing with weird and broken devices.

On Tue, Feb 27, 2018 at 11:08:20AM +0100, Rafa? Mi?ecki
wrote:
> I've problem when using OpenWrt/LEDE on a home router with
Broadcom's
> FullMAC WiFi chipset.
Hi Rafa?,

Thanks for reporting this issue!
> Can you see any solution for this problem? Is that an option to stop
> multicast-to-unicast from touching 802.11f packets? Some other ideas?
> Obviously I can't modify Broadcom's firmware and drop that
obsoleted
> standard.
Just to avoid some potential confusion: This is more an issue of
hairpinning than it is an issue of multicast-to-unicast per se,
right?

That is, if you set this to 0 manually:
/sys/class/net/<ap-iface>/brport/multicast_to_unicast
Then the issue still occurs, right?

Regards, Linus

On 02/27/2018 11:08 AM, Rafa? Mi?ecki wrote:
> I've problem when using OpenWrt/LEDE on a home router with
Broadcom's
> FullMAC WiFi chipset.
>
>
> First of all OpenWrt/LEDE uses bridge interface for LAN network with:
> 1) IFLA_BRPORT_MCAST_TO_UCAST
> 2) Clients isolation in hostapd
> 3) Hairpin mode enabled
>
> For more details please see Linus's patch description:
> https://patchwork.kernel.org/patch/9530669/
> and maybe hairpin mode patch:
> https://lwn.net/Articles/347344/
>
> Short version: in that setup packets received from a bridged wireless
> interface can be handled back to it for transmission.
>
>
> Now, Broadcom's firmware for their FullMAC chipsets in AP mode
> supports an obsoleted 802.11f AKA IAPP standard. It's a roaming
> standard that was replaced by 802.11r.
>
> Whenever a new station associates, firmware generates a packet like:
> ff ff ff ff  ff ff ec 10  7b 5f ?? ??  00 06 00 01  af 81 01 00
> (just masked 2 bytes of my MAC)
>
> For mode details you can see discussion in my brcmfmac patch thread:
> https://patchwork.kernel.org/patch/10191451/
>
>
> The problem is that bridge (in setup as above) handles such a packet
> back to the device.
>
> That makes Broadcom's FullMAC firmware believe that a given station
> just connected to another AP in a network (which doesn't even exist).
> As a result firmware immediately disassociates that station. It's
> simply impossible to connect to the router. Every association is
> followed by immediate disassociation.
>
>
> Can you see any solution for this problem? Is that an option to stop
> multicast-to-unicast from touching 802.11f packets? Some other ideas?
> Obviously I can't modify Broadcom's firmware and drop that
obsoleted
> standard.
I kept thinking about this for a bit more. We probably have to start
with deciding what to blame for this problem. I see two options I'll
describe below.

Please share your opinions on this.


1) Blame brcmfmac fw for implementing 802.11f

In that case we should add workaround to brcmfmac driver to drop/ignore
802.11f packets. A slightly improved version of my
[PATCH] brcmfmac: detect & reject faked packet generated by a firmware
https://patchwork.kernel.org/patch/10191451/
should work.


2) Blame bridge + mcast-to-ucast + hairpin for 802.11f incompatibility

If we agree that 802.11f support in FullMAC firmware is acceptable, then
we have to make sure Linux's bridge doesn't break it by passing 802.11f
(broadcast) frames back to the source interface. That would require a
check like in below diff + proper code for handling such packets. I'm
afraid I'm not familiar with bridge code enough to complete that.

diff --git a/net/bridge/br_input.c b/net/bridge/br_input.c
index edae702..9e5d6ea 100644
--- a/net/bridge/br_input.c
+++ b/net/bridge/br_input.c
@@ -126,6 +126,27 @@ static void br_do_proxy_arp(struct sk_buff *skb, struct
net_bridge *br,
  	}
  }

+static bool br_skb_is_iapp_add_packet(struct sk_buff *skb)
+{
+	const u8 iapp_add_packet[6] __aligned(2) = {
+		0x00, 0x01, 0xaf, 0x81, 0x01, 0x00,
+	};
+#if !defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)
+	const u16 *a = (const u16 *)skb->data;
+	const u16 *b = (const u16 *)iapp_add_packet;
+#endif
+
+	if (skb->len != 6)
+		return false;
+
+#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)
+	return !(((*(const u32 *)skb->data) ^ (*(const u32 *)iapp_add_packet)) |
+	         ((*(const u16 *)(skb->data + 4)) ^ (*(const u16 *)(iapp_add_packet
+ 4))));
+#else
+	return !((a[0] ^ b[0]) | (a[1] ^ b[1]) | (a[2] ^ b[2]));
+#endif
+}
+
  /* note: already called with rcu_read_lock */
  int br_handle_frame_finish(struct net *net, struct sock *sk, struct sk_buff
*skb)
  {
@@ -155,6 +176,8 @@ int br_handle_frame_finish(struct net *net, struct sock *sk,
struct sk_buff *skb
  	if (is_multicast_ether_addr(dest)) {
  		/* by definition the broadcast is also a multicast address */
  		if (is_broadcast_ether_addr(dest)) {
+			if (br_skb_is_iapp_add_packet(skb))
+				pr_warn("This packet should not be passed back to the source
interface!\n");
  			pkt_type = BR_PKT_BROADCAST;
  			local_rcv = true;
  		} else {

On Mon, 12 Mar 2018 23:42:48 +0100
Rafa? Mi?ecki <zajec5 at gmail.com> wrote:
> 2) Blame bridge + mcast-to-ucast + hairpin for 802.11f incompatibility
> 
> If we agree that 802.11f support in FullMAC firmware is acceptable, then
> we have to make sure Linux's bridge doesn't break it by passing
802.11f
> (broadcast) frames back to the source interface. That would require a
> check like in below diff + proper code for handling such packets. I'm
> afraid I'm not familiar with bridge code enough to complete that.
> 
> diff --git a/net/bridge/br_input.c b/net/bridge/br_input.c
> index edae702..9e5d6ea 100644
> --- a/net/bridge/br_input.c
> +++ b/net/bridge/br_input.c
> @@ -126,6 +126,27 @@ static void br_do_proxy_arp(struct sk_buff *skb,
struct net_bridge *br,
>   	}
>   }
> 
> +static bool br_skb_is_iapp_add_packet(struct sk_buff *skb)
> +{
> +	const u8 iapp_add_packet[6] __aligned(2) = {
> +		0x00, 0x01, 0xaf, 0x81, 0x01, 0x00,
> +	};
> +#if !defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)
> +	const u16 *a = (const u16 *)skb->data;
> +	const u16 *b = (const u16 *)iapp_add_packet;
> +#endif
> +
> +	if (skb->len != 6)
> +		return false;
> +
> +#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)
> +	return !(((*(const u32 *)skb->data) ^ (*(const u32 *)iapp_add_packet))
|
> +	         ((*(const u16 *)(skb->data + 4)) ^ (*(const u16
*)(iapp_add_packet + 4))));
> +#else
> +	return !((a[0] ^ b[0]) | (a[1] ^ b[1]) | (a[2] ^ b[2]));
> +#endif
> +}
> +
>   /* note: already called with rcu_read_lock */
>   int br_handle_frame_finish(struct net *net, struct sock *sk, struct
sk_buff *skb)
>   {
> @@ -155,6 +176,8 @@ int br_handle_frame_finish(struct net *net, struct sock
*sk, struct sk_buff *skb
>   	if (is_multicast_ether_addr(dest)) {
>   		/* by definition the broadcast is also a multicast address */
>   		if (is_broadcast_ether_addr(dest)) {
> +			if (br_skb_is_iapp_add_packet(skb))
> +				pr_warn("This packet should not be passed back to the source
interface!\n");
>   			pkt_type = BR_PKT_BROADCAST;
>   			local_rcv = true;
>   		} else {

Don't like bridge doing special case code for magic received values directly
in input path.
Really needs to be generic which is why I suggested ebtables.

On 2018-02-27 11:08, Rafa? Mi?ecki wrote:
> I've problem when using OpenWrt/LEDE on a home router with
Broadcom's
> FullMAC WiFi chipset.
> 
> 
> First of all OpenWrt/LEDE uses bridge interface for LAN network with:
> 1) IFLA_BRPORT_MCAST_TO_UCAST
> 2) Clients isolation in hostapd
> 3) Hairpin mode enabled
> 
> For more details please see Linus's patch description:
> https://patchwork.kernel.org/patch/9530669/
> and maybe hairpin mode patch:
> https://lwn.net/Articles/347344/
> 
> Short version: in that setup packets received from a bridged wireless
> interface can be handled back to it for transmission.
> 
> 
> Now, Broadcom's firmware for their FullMAC chipsets in AP mode
> supports an obsoleted 802.11f AKA IAPP standard. It's a roaming
> standard that was replaced by 802.11r.
> 
> Whenever a new station associates, firmware generates a packet like:
> ff ff ff ff  ff ff ec 10  7b 5f ?? ??  00 06 00 01  af 81 01 00
> (just masked 2 bytes of my MAC)
> 
> For mode details you can see discussion in my brcmfmac patch thread:
> https://patchwork.kernel.org/patch/10191451/
> 
> 
> The problem is that bridge (in setup as above) handles such a packet
> back to the device.
> 
> That makes Broadcom's FullMAC firmware believe that a given station
> just connected to another AP in a network (which doesn't even exist).
> As a result firmware immediately disassociates that station. It's
> simply impossible to connect to the router. Every association is
> followed by immediate disassociation.
> 
> 
> Can you see any solution for this problem? Is that an option to stop
> multicast-to-unicast from touching 802.11f packets? Some other ideas?
> Obviously I can't modify Broadcom's firmware and drop that
obsoleted
> standard.
Let's look at it from a different angle: Since these packets are
forwarded as normal packets by the bridge, and the Broadcom firmware
reacts to them in this nasty way, that's basically local DoS security
issue. In my opinion that matters a lot more than having support for an
obsolete feature that almost nobody will ever want to use.

I think the right approach to deal with this issue is to drop these
garbage packets in both the receive and transmit path of brcmfmac.

- Felix

[resent with fixed typo in linux-wireless address]

On 2018-02-27 11:08, Rafa? Mi?ecki wrote:
> I've problem when using OpenWrt/LEDE on a home router with
Broadcom's
> FullMAC WiFi chipset.
> 
> 
> First of all OpenWrt/LEDE uses bridge interface for LAN network with:
> 1) IFLA_BRPORT_MCAST_TO_UCAST
> 2) Clients isolation in hostapd
> 3) Hairpin mode enabled
> 
> For more details please see Linus's patch description:
> https://patchwork.kernel.org/patch/9530669/
> and maybe hairpin mode patch:
> https://lwn.net/Articles/347344/
> 
> Short version: in that setup packets received from a bridged wireless
> interface can be handled back to it for transmission.
> 
> 
> Now, Broadcom's firmware for their FullMAC chipsets in AP mode
> supports an obsoleted 802.11f AKA IAPP standard. It's a roaming
> standard that was replaced by 802.11r.
> 
> Whenever a new station associates, firmware generates a packet like:
> ff ff ff ff  ff ff ec 10  7b 5f ?? ??  00 06 00 01  af 81 01 00
> (just masked 2 bytes of my MAC)
> 
> For mode details you can see discussion in my brcmfmac patch thread:
> https://patchwork.kernel.org/patch/10191451/
> 
> 
> The problem is that bridge (in setup as above) handles such a packet
> back to the device.
> 
> That makes Broadcom's FullMAC firmware believe that a given station
> just connected to another AP in a network (which doesn't even exist).
> As a result firmware immediately disassociates that station. It's
> simply impossible to connect to the router. Every association is
> followed by immediate disassociation.
> 
> 
> Can you see any solution for this problem? Is that an option to stop
> multicast-to-unicast from touching 802.11f packets? Some other ideas?
> Obviously I can't modify Broadcom's firmware and drop that
obsoleted
> standard.
Let's look at it from a different angle: Since these packets are
forwarded as normal packets by the bridge, and the Broadcom firmware
reacts to them in this nasty way, that's basically local DoS security
issue. In my opinion that matters a lot more than having support for an
obsolete feature that almost nobody will ever want to use.

I think the right approach to deal with this issue is to drop these
garbage packets in both the receive and transmit path of brcmfmac.

- Felix