freebsd stable - Apr 2014 - Some gruesome moments with performance of FreeBSD at over 20K interfaces

> On Wed, 9 Apr 2014, Vladislav Prodan wrote:
> 
> VP>b) Service bsnmpd started at 12K interfaces, but immediately loaded
CPU
> VP>at 80-100%
> 
> I could imagine that this is because of the statistics polling. bsnmp 
> implements 64-bit interface statistics but we have only 32-bit statistics 
> in the kernel. So it polls the kernel statistics for each interface on a 
> rate that ensures that 32-bit don't overflow. If the interfaces are
GBit
> or, worse, 10GBit interfaces the polling rate is rather high (in the order 
> of seconds).
> 
> You should either make sure that the interfaces report sensible bitrates 
> (I doubt that 20k interfaces could all be GBit interfaces) or force a
slower
> polling interval by setting begemotIfForcePoll.0 to some large value.
> 
> harti
> 
Thanks for the tip.

At least 10 interfaces to be 1Gb, and the rest no more than 50M.
BegemotIfForcePoll parameter in this case a little help, you will be forced to
stand another value for Gigabit Interface begemotIfForcePoll ...

-- 
Vladislav V. Prodan 
System & Network Administrator 
http://support.od.ua 
+380 67 4584408, +380 99 4060508
VVP88-RIPE

On Thu, 10 Apr 2014, Vladislav Prodan wrote:

VP>> On Wed, 9 Apr 2014, Vladislav Prodan wrote:
VP>> 
VP>> VP>b) Service bsnmpd started at 12K interfaces, but immediately
loaded CPU
VP>> VP>at 80-100%
VP>> 
VP>> I could imagine that this is because of the statistics polling. bsnmp
VP>> implements 64-bit interface statistics but we have only 32-bit
statistics
VP>> in the kernel. So it polls the kernel statistics for each interface
on a
VP>> rate that ensures that 32-bit don't overflow. If the interfaces
are GBit
VP>> or, worse, 10GBit interfaces the polling rate is rather high (in the
order
VP>> of seconds).
VP>> 
VP>> You should either make sure that the interfaces report sensible
bitrates
VP>> (I doubt that 20k interfaces could all be GBit interfaces) or force a
slower
VP>> polling interval by setting begemotIfForcePoll.0 to some large value.
VP>> 
VP>> harti
VP>> 
VP>
VP>Thanks for the tip.
VP>
VP>At least 10 interfaces to be 1Gb, and the rest no more than 50M.
VP>BegemotIfForcePoll parameter in this case a little help, you will be
forced to stand another value for Gigabit Interface begemotIfForcePoll ...

Yeah. There is only one parameter.

You are running -STABLE, right? On current the statistics are 64-bit 
(I wonder whether the operation on these values is automatically atomic on 
all our platforms, though).

harti

Another note related to Q-in-Q.

You would probably be better of creating standard vlans for the first vlan
layer and use ng_vlan for the second++ part of the Q-in-Q on top of the
first ones.
This also give better usability and will speedup a bit your times.


On Thu, Apr 10, 2014 at 1:22 PM, Hartmut Brandt
<hartmut.brandt@dlr.de>wrote:
> On Thu, 10 Apr 2014, Vladislav Prodan wrote:
>
> VP>> On Wed, 9 Apr 2014, Vladislav Prodan wrote:
> VP>>
> VP>> VP>b) Service bsnmpd started at 12K interfaces, but
immediately
> loaded CPU
> VP>> VP>at 80-100%
> VP>>
> VP>> I could imagine that this is because of the statistics polling.
bsnmp
> VP>> implements 64-bit interface statistics but we have only 32-bit
> statistics
> VP>> in the kernel. So it polls the kernel statistics for each
interface
> on a
> VP>> rate that ensures that 32-bit don't overflow. If the
interfaces are
> GBit
> VP>> or, worse, 10GBit interfaces the polling rate is rather high (in
the
> order
> VP>> of seconds).
> VP>>
> VP>> You should either make sure that the interfaces report sensible
> bitrates
> VP>> (I doubt that 20k interfaces could all be GBit interfaces) or
force a
> slower
> VP>> polling interval by setting begemotIfForcePoll.0 to some large
value.
> VP>>
> VP>> harti
> VP>>
> VP>
> VP>Thanks for the tip.
> VP>
> VP>At least 10 interfaces to be 1Gb, and the rest no more than 50M.
> VP>BegemotIfForcePoll parameter in this case a little help, you will be
> forced to stand another value for Gigabit Interface begemotIfForcePoll ...
>
> Yeah. There is only one parameter.
>
> You are running -STABLE, right? On current the statistics are 64-bit
> (I wonder whether the operation on these values is automatically atomic on
> all our platforms, though).
>
> harti
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
"freebsd-net-unsubscribe@freebsd.org"
>


-- 
Ermal
_______________________________________________
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to
"freebsd-stable-unsubscribe@freebsd.org"