Harti Brandt
2014-Apr-10 10:08 UTC
Some gruesome moments with performance of FreeBSD at over 20K interfaces
On Wed, 9 Apr 2014, Vladislav Prodan wrote: VP>b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU VP>at 80-100% I could imagine that this is because of the statistics polling. bsnmp implements 64-bit interface statistics but we have only 32-bit statistics in the kernel. So it polls the kernel statistics for each interface on a rate that ensures that 32-bit don't overflow. If the interfaces are GBit or, worse, 10GBit interfaces the polling rate is rather high (in the order of seconds). You should either make sure that the interfaces report sensible bitrates (I doubt that 20k interfaces could all be GBit interfaces) or force a slower polling interval by setting begemotIfForcePoll.0 to some large value. harti
Vladislav Prodan
2014-Apr-10 11:05 UTC
Some gruesome moments with performance of FreeBSD at over 20K interfaces
> On Wed, 9 Apr 2014, Vladislav Prodan wrote: > > VP>b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU > VP>at 80-100% > > I could imagine that this is because of the statistics polling. bsnmp > implements 64-bit interface statistics but we have only 32-bit statistics > in the kernel. So it polls the kernel statistics for each interface on a > rate that ensures that 32-bit don't overflow. If the interfaces are GBit > or, worse, 10GBit interfaces the polling rate is rather high (in the order > of seconds). > > You should either make sure that the interfaces report sensible bitrates > (I doubt that 20k interfaces could all be GBit interfaces) or force a slower > polling interval by setting begemotIfForcePoll.0 to some large value. > > harti >Thanks for the tip. At least 10 interfaces to be 1Gb, and the rest no more than 50M. BegemotIfForcePoll parameter in this case a little help, you will be forced to stand another value for Gigabit Interface begemotIfForcePoll ... -- Vladislav V. Prodan System & Network Administrator http://support.od.ua +380 67 4584408, +380 99 4060508 VVP88-RIPE
Hartmut Brandt
2014-Apr-10 11:22 UTC
Re[2]: Some gruesome moments with performance of FreeBSD at over 20K interfaces
On Thu, 10 Apr 2014, Vladislav Prodan wrote: VP>> On Wed, 9 Apr 2014, Vladislav Prodan wrote: VP>> VP>> VP>b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU VP>> VP>at 80-100% VP>> VP>> I could imagine that this is because of the statistics polling. bsnmp VP>> implements 64-bit interface statistics but we have only 32-bit statistics VP>> in the kernel. So it polls the kernel statistics for each interface on a VP>> rate that ensures that 32-bit don't overflow. If the interfaces are GBit VP>> or, worse, 10GBit interfaces the polling rate is rather high (in the order VP>> of seconds). VP>> VP>> You should either make sure that the interfaces report sensible bitrates VP>> (I doubt that 20k interfaces could all be GBit interfaces) or force a slower VP>> polling interval by setting begemotIfForcePoll.0 to some large value. VP>> VP>> harti VP>> VP> VP>Thanks for the tip. VP> VP>At least 10 interfaces to be 1Gb, and the rest no more than 50M. VP>BegemotIfForcePoll parameter in this case a little help, you will be forced to stand another value for Gigabit Interface begemotIfForcePoll ... Yeah. There is only one parameter. You are running -STABLE, right? On current the statistics are 64-bit (I wonder whether the operation on these values is automatically atomic on all our platforms, though). harti _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"