Linux Ethernet Bridging - Aug 2015 - [Bridge] [PATCH net-next v2] bridge: vlan: allow to suppress local mac install for all vlans

> On Aug 25, 2015, at 7:42 PM, David Miller <davem at davemloft.net>
wrote:
> 
> From: Nikolay Aleksandrov <razor at blackwall.org>
> Date: Tue, 25 Aug 2015 17:34:55 -0700
> 
>> From: Nikolay Aleksandrov <nikolay at cumulusnetworks.com>
>> 
>> This patch adds a new knob that, when enabled, allows to suppress the
>> installation of local fdb entries in newly created vlans. This could
>> pose a big scalability issue if we have a large number of ports and a
>> large number of vlans, e.g. in a 48 port device with 2000 vlans these
>> entries easily go up to 96000.
>> Note that packets for these macs are still received properly because
they
>> are added in vlan 0 as "own" macs and referenced when fdb
lookup by vlan
>> results in a miss.
>> Also note that vlan membership of ingress port and the bridge device
>> as egress are still being correctly enforced.
>> 
>> The default (0/off) is keeping the current behaviour.
>> 
>> Based on a patch by Wilson Kok (wkok at cumulusnetworks.com).
>> 
>> Signed-off-by: Nikolay Aleksandrov <nikolay at
cumulusnetworks.com>
>> ---
>> v2: Triple checked the timezone
> 
> I'd rather we fix the essence of the scalability problem than add
> more spaghetti code to the various bridge paths.
> 
> Can we make the fdb entries smaller?
> 
> Can we enhance how we store such local entries such that they live in
> a compact datastructure?  Perhaps the FDB can consist of a very dense
> lookup mechanism for local stuff sitting alongside the current table.
Certainly, that should be done and I will look into it, but the essence of this
patch
is a bit different. The problem here is not the size of the fdb entries, it?s
more the
number of them - having 96000 entries (even if they were 1 byte ones) is just
way
too much especially when the fdb hash size is small and static. We could work on
making
it dynamic though, but still these type of local entries per vlan per port can
easily be avoided
with this option.

From: Nikolay Aleksandrov <nikolay at cumulusnetworks.com>
Date: Tue, 25 Aug 2015 22:28:16 -0700
> Certainly, that should be done and I will look into it, but the
> essence of this patch is a bit different. The problem here is not
> the size of the fdb entries, it?s more the number of them - having
> 96000 entries (even if they were 1 byte ones) is just way too much
> especially when the fdb hash size is small and static. We could work
> on making it dynamic though, but still these type of local entries
> per vlan per port can easily be avoided with this option.
96000 bits can be stored in 12k.  Get where I'm going with this?

Look at the problem sideways.

>>
>> I'd rather we fix the essence of the scalability problem than add
>> more spaghetti code to the various bridge paths.
>>
>> Can we make the fdb entries smaller?
>>
>> Can we enhance how we store such local entries such that they live in
>> a compact datastructure?  Perhaps the FDB can consist of a very dense
>> lookup mechanism for local stuff sitting alongside the current table.
>
> Certainly, that should be done and I will look into it, but the essence of
this patch
> is a bit different. The problem here is not the size of the fdb entries,
it?s more the
> number of them - having 96000 entries (even if they were 1 byte ones) is
just way
> too much especially when the fdb hash size is small and static. We could
work on making
> it dynamic though, but still these type of local entries per vlan per port
can easily be avoided
> with this option.
>
I was wondering if it is possible to assign a vlan bitmap for the FDB
entry, instead of replicating the entry for each vlan. ( I believe
Roopa has done something similar, but not so sure). This means that
the number of FDB entries remain static for any number of vlans.

I guess its more complicated than it sounds, but just wanted to know
if its feasible at all.

Thanks
Vissu
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev"
in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 08/26/2015 02:10 AM, B Viswanath wrote:
>>>
>>> I'd rather we fix the essence of the scalability problem than
add
>>> more spaghetti code to the various bridge paths.
>>>
>>> Can we make the fdb entries smaller?
>>>
>>> Can we enhance how we store such local entries such that they live
in
>>> a compact datastructure?  Perhaps the FDB can consist of a very
dense
>>> lookup mechanism for local stuff sitting alongside the current
table.
>>
>> Certainly, that should be done and I will look into it, but the essence
of this patch
>> is a bit different. The problem here is not the size of the fdb
entries, it?s more the
>> number of them - having 96000 entries (even if they were 1 byte ones)
is just way
>> too much especially when the fdb hash size is small and static. We
could work on making
>> it dynamic though, but still these type of local entries per vlan per
port can easily be avoided
>> with this option.
>>
> 
> I was wondering if it is possible to assign a vlan bitmap for the FDB
> entry, instead of replicating the entry for each vlan. ( I believe
> Roopa has done something similar, but not so sure). This means that
> the number of FDB entries remain static for any number of vlans.
> 
> I guess its more complicated than it sounds, but just wanted to know
> if its feasible at all.
I've actually had this done in one of the earlier attempts.  The issue was
how
to compress it because there was absolutely no gain if you have a sparse vlan
bitmap.

I even tried doing something along the lines of vlan_group array, but that can
explode to full size almost as fast.

What actually worked better was a hash table of vlans where each entry in the
table
contained a bunch of data one of which was a list of fdbs for a given vlan.   It
didn't replicate fdbs but simply referenced the ones we cared about and
bumped the ref.

However, this made vlan look-ups slower since we now had a hash instead of a
bitmap lookup
and Stephen rejected it.

-vlad
> 
> Thanks
> Vissu
> 
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe
netdev" in
>> the body of a message to majordomo at vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html