Greetings, I''m trying to work out whether flow-tools will allow me to retrieve (or calculate) a second-accurate flow start-time in seconds since the UNIX epoch. If my understanding is correct, and refering to the NetFlow v9 specification along with store.h, AGENT_INFO contains time_sec & time_nanosec, but these appear to always take the same value as RECV_TIME.recv_sec. I want to calculate a flows start and stop times relative to unix epoch rather than the devices uptime. Would the following give me what I''m looking for? actual_flows_start (AGENT_INFO.time_sec - 100*AGENT_INFO.sys_uptime_ms) + FLOW_TIMES.flows_start Is there a more sane/sensible way? On a semi-related note, I''ve locally patched flowd-reader to support export to SQLite to facilitate further analysis. Would anyone else be interested in my cleaning up my patches and submitting them? Robin -- Robin Breathe, Computer Services, Oxford Brookes University, Oxford, UK rbreathe at brookes.ac.uk Tel: +44 1865 483685 Fax: +44 1865 483073 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 187 bytes Desc: OpenPGP digital signature Url : http://lists.mindrot.org/pipermail/netflow-tools/attachments/20050926/9051bdd9/attachment.bin
Robin Breathe wrote:> Greetings, > > I''m trying to work out whether flow-tools will allow me to retrieve (or > calculate) a second-accurate flow start-time in seconds since the UNIX > epoch. > > If my understanding is correct, and refering to the NetFlow v9 > specification along with store.h, AGENT_INFO contains time_sec & > time_nanosec, but these appear to always take the same value as > RECV_TIME.recv_sec.They are in fact different: recv_sec is set by flowd, but time_sec is set by the flow exported. If they are the same, then it is because your clocks are in sync and there was no lag between flow expiry and export. On a Cisco, for example, the first flows in an export packet usually have the largest delta between recv_sec, as they were expired and subsequently queued first. On a busy router, there may not be much of a lag though. On the other hand, if you are using something like softflowd that exports flows as soon as they expire, then the only difference that you will see is the difference between the exporter and the collector''s clock, or zero if they are on the same host.> I want to calculate a flows start and stop times > relative to unix epoch rather than the devices uptime. > > Would the following give me what I''m looking for? > > actual_flows_start > (AGENT_INFO.time_sec - 100*AGENT_INFO.sys_uptime_ms) > + FLOW_TIMES.flows_startShouldn''t this be: time_sec - (sys_uptime_ms / 1000) + flow_start ?> Is there a more sane/sensible way?I don''t think so. BTW this is already in the TODO list, so it would be a welcome addition even if it just a helper macro or two in store.h.> On a semi-related note, I''ve locally patched flowd-reader to support > export to SQLite to facilitate further analysis. Would anyone else be > interested in my cleaning up my patches and submitting them?Probably not as part of flowd-reader, but a separate tool (to live in the tools/ subdirectory) would be most welcome. There is already a Perl script to do just this there. -d
The actual_flows_start equation I gave before was total junk. Does the following look more reasonable? unix_flow_start = (ntohl(flow->ainfo.time_sec) - ntohl(flow->ainfo.sys_uptime_ms)/1000) + ntohl(flow->ftimes.flow_start)/1000; This appears to give relatively sensible results, but is it correct? Robin -- Robin Breathe, Computer Services, Oxford Brookes University, Oxford, UK rbreathe at brookes.ac.uk Tel: +44 1865 483685 Fax: +44 1865 483073 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 187 bytes Desc: OpenPGP digital signature Url : http://lists.mindrot.org/pipermail/netflow-tools/attachments/20050926/37c5f282/attachment.bin
Damien Miller wrote:> ...snip... > Shouldn''t this be: > > time_sec - (sys_uptime_ms / 1000) + flow_start > > ?Pretty much (see my other post).>> Is there a more sane/sensible way? > > I don''t think so. > > BTW this is already in the TODO list, so it would be a welcome addition > even if it just a helper macro or two in store.h.I''ll have a look.>> On a semi-related note, I''ve locally patched flowd-reader to support >> export to SQLite to facilitate further analysis. Would anyone else be >> interested in my cleaning up my patches and submitting them? > > Probably not as part of flowd-reader, but a separate tool (to live in > the tools/ subdirectory) would be most welcome. There is already a > Perl script to do just this there.Yup, I had trouble getting the perl module to work under solaris9/sparc, so decided to patch it into flowd-reader (since I could then also exploit its existing filter code). A feature of my SQLite exporter is that it will merge both multiple instances of a flow and the two sides of a bidirectional dataflow into a single row (by grouping on the src_addr:src_port/dst_addr:dst_port/proto tuple). I have another utility to merge flows in adjacent exports into a single table. However, at present both of these utilities only work on a fixed set of columns: start_time, duration, {src,dst}_{addr,port}, proto, {in,out}_{octets,packets} Robin -- Robin Breathe, Computer Services, Oxford Brookes University, Oxford, UK rbreathe at brookes.ac.uk Tel: +44 1865 483685 Fax: +44 1865 483073 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 187 bytes Desc: OpenPGP digital signature Url : http://lists.mindrot.org/pipermail/netflow-tools/attachments/20050926/af9b08c3/attachment.bin