Jeff Saxe
2008-Mar-28 15:12 UTC
[netflow-tools] flowd is terrific! What do the continuous syslog messages mean?
My apologies for first sending this directly to the author instead of to a proper mailing list, which he took the time to set up. I am resending it to the list. Good day! I''m a network engineer, smart guy, and Perl hacker at a smallish ISP in Charlottesville, Virginia, USA, and I''m trying to use flowd to put together a clever little client billing system. I believe everything is working fine, but when I turn on several routers'' and several interfaces'' worth of NetFlow packets toward this (reasonably powerful) Linux box at the same time, I get syslog messages like... Mar 26 20:13:27 chance flowd[17102]: Received max number of packets (512) on fd 3 Mar 26 20:13:27 chance flowd[17102]: Valid netflow v.5 packet 30 flows Mar 26 20:13:27 chance flowd[17102]: Received max number of packets (512) on fd 3 Mar 26 20:13:27 chance flowd[17102]: Valid netflow v.5 packet 30 flows Mar 26 20:13:27 chance flowd[17102]: Valid netflow v.5 packet 30 flows Mar 26 20:13:27 chance flowd[17102]: Valid netflow v.5 packet 30 flows Does this mean that actual NetFlow data are being discarded because they are arriving too fast? I should warn you that I''m using a lot of flowd.conf "accept" rules (on the order of 1,300, and more coming next week), so is that a problem? I mainly want to know if all the data are being collected or not; if they are, I will recompile the code to just suppress this warning message, but if it''s a real problem, I''d like to know if you have any suggestions for not dropping packets. I''ve attached the current flowd.conf in case you are interested, and if you want to see the Perl code or MySQL table structure behind the automated rule generation, I can show you that, too. ? If you don''t have time to help a stranger with this for free, I understand; just please tell me if the code is throwing away flows or not. Thanks very much, sir! -- Jeff Saxe, Network Engineer Blue Ridge InternetWorks, Charlottesville, VA CCIE # 9376 434-817-0707 ext. 2024 (work) / 434-882-3508 (cell) / JSaxe at briworks.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.mindrot.org/pipermail/netflow-tools/attachments/20080328/2f6b6280/attachment-0002.html -------------- next part -------------- A non-text attachment was scrubbed... Name: flowd.conf Type: application/octet-stream Size: 263352 bytes Desc: not available Url : http://lists.mindrot.org/pipermail/netflow-tools/attachments/20080328/2f6b6280/attachment-0001.obj -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.mindrot.org/pipermail/netflow-tools/attachments/20080328/2f6b6280/attachment-0003.html
Damien Miller
2008-Apr-23 02:02 UTC
[netflow-tools] flowd is terrific! What do the continuous syslog messages mean?
Hi,
Sorry for taking a little while to get back to you.
Could you please try this patch? (The messages are harmless)
Index: flowd.c
==================================================================RCS file:
/var/cvs/flowd/flowd.c,v
retrieving revision 1.77
diff -u -p -r1.77 flowd.c
--- flowd.c 24 Oct 2007 01:04:10 -0000 1.77
+++ flowd.c 23 Apr 2008 02:00:52 -0000
@@ -1185,7 +1185,7 @@ receive_many(struct flowd_config *conf,
for (i = 0; i < INPUT_MAX_PACKET_PER_FD; i++) {
if (receive_packet(conf, peers, net_fd) == 0) {
- syslog(LOG_DEBUG, "Received max number of packets "
+ logit(LOG_DEBUG, "Received max number of packets "
"(%d) on fd %d", INPUT_MAX_PACKET_PER_FD, net_fd);
return;
}
On Fri, 28 Mar 2008, Jeff Saxe wrote:
> My apologies for first sending this directly to the author instead of to a
> proper mailing list, which he took the time to set up. I am resending it to
> the list.
>
>
> Good day! I''m a network engineer, smart guy, and Perl hacker at a
smallish ISP
> in Charlottesville, Virginia, USA, and I''m trying to use flowd to
put together
> a clever little client billing system. I believe everything is working
fine,
> but when I turn on several routers'' and several
interfaces'' worth of NetFlow
> packets toward this (reasonably powerful) Linux box at the same time, I get
> syslog messages like...
>
>
> Mar 26 20:13:27 chance flowd[17102]: Received max number of packets (512)
on
> fd 3
> Mar 26 20:13:27 chance flowd[17102]: Valid netflow v.5 packet 30 flows
> Mar 26 20:13:27 chance flowd[17102]: Received max number of packets (512)
on
> fd 3
> Mar 26 20:13:27 chance flowd[17102]: Valid netflow v.5 packet 30 flows
> Mar 26 20:13:27 chance flowd[17102]: Valid netflow v.5 packet 30 flows
> Mar 26 20:13:27 chance flowd[17102]: Valid netflow v.5 packet 30 flows
>
> Does this mean that actual NetFlow data are being discarded because they
are
> arriving too fast? I should warn you that I''m using a lot of
flowd.conf
> "accept" rules (on the order of 1,300, and more coming next
week), so is that
> a problem? I mainly want to know if all the data are being collected or
not;
> if they are, I will recompile the code to just suppress this warning
message,
> but if it''s a real problem, I''d like to know if you have
any suggestions for
> not dropping packets. I''ve attached the current flowd.conf in case
you are
> interested, and if you want to see the Perl code or MySQL table structure
> behind the automated rule generation, I can show you that, too.
>
> ?
>
> If you don''t have time to help a stranger with this for free, I
understand;
> just please tell me if the code is throwing away flows or not. Thanks very
> much, sir!
>
> -- Jeff Saxe, Network Engineer
> Blue Ridge InternetWorks, Charlottesville, VA
> CCIE # 9376
> 434-817-0707 ext. 2024 (work) / 434-882-3508 (cell) / JSaxe at
briworks.com
>
>
>
>