Todor Dragnev
2006-Jan-06 01:08 UTC
[jonathan.glass@oit.gatech.edu: Re: [Flow-tools] Memory leak ?]
Can someone help with this ? ----- Forwarded message from Jonathan Glass <jonathan.glass@oit.gatech.edu> ----- From: Jonathan Glass <jonathan.glass@oit.gatech.edu> To: Mike Hunter <mhunter@ack.berkeley.edu> Cc: Todor Dragnev <todor@linuxfan.org>, flow-tools@list.splintered.net Subject: Re: [Flow-tools] Memory leak ? Date: Thu, 05 Jan 2006 09:27:10 -0500 User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923) Message-ID: <43BD2CBE.4060403@oit.gatech.edu> Mike Hunter wrote:>On Jan 04, "Todor Dragnev" wrote: > > >>Hello list, >> >>I use flow-control from about 1 week. My OS is FreeBSD 6.0-RELEASE #0 >>for AMD64. All works fine but yesterday I found this in dmesg: >> >>Jan 4 21:30:58 katana kernel: swap_pager_getswapspace(16): failed >>Jan 4 21:30:58 katana kernel: swap_pager_getswapspace(12): failed >>Jan 4 21:30:58 katana kernel: swap_pager_getswapspace(6): failed >> >>I have 1024MB RAM and 2GB swapspace >>On this PC I have squid-cache running, but when check which program use >>more >>memory I found this: >> >>last pid: 49917; load averages: 0.94, 0.96, 0.79 up 1+05:58:05 >>21:58:29 >>187 processes: 2 running, 184 sleeping, 1 lock >>CPU states: 5.5% user, 0.0% nice, 25.6% system, 21.3% interrupt, 47.6% >>idle >>Mem: 678M Active, 59M Inact, 166M Wired, 36M Cache, 111M Buf, 1636K Free >>Swap: 2048M Total, 2048M Used, K Free, 100% Inuse, 4K In, 32K Out >> >>PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND >>518 root 1 96 0 2648M 104M select 0:33 0.00% flow-capture >>635 www 1 4 0 18504K 8908K kqread 0:22 0.00% thttpd >> >> >>My starting line for flow-capture is: >> >>/usr/local/bin/flow-capture -p /var/run/flow-capture.pid -n 287 -N 0 -w >>/var/log/netflows/ -S 5 /127.0.0.1/8899 >> >>Is that huge memory usage is memory leak or I do something wrong ? > > >There could be a 64-bit problem. Our setup has some 32-bit collectors that >feed (among other things) a 64-bit cruncher. Building out the 64-bit >cruncher lead me to finding some 32-bit vs. 64-bit bugs: > >http://mailman.splintered.net/pipermail/flow-tools/2004-December/002501.html > >But I haven't had to deal with flow-capture on 64-bit.... > >Is anybody out there running flow-capture on amd64/x86_64 or another 64 >bit platform? > >I think the easiest way to start looking at this would be to run >flow-capture under a memory debugger of some sort, like efence >(Electric Fence Malloc Debugger). > >MikeI'm running flow-capture on AMD64 on Fedora Core 3 with no problems. The only issue I run into is lack of disk space! Sometimes 50GB is not enough! Linux netflow 2.6.10-1.766_FC3smp #1 SMP Wed Feb 9 23:17:48 EST 2005 x86_64 x86_64 x86_64 GNU/Linux /usr/local/netflow/bin/flow-capture -d1 -w /var/netflow/ -z3 -V5 -N3 -n96 -S1 -E48G 127.0.0.1/0/2057 > /var/netflow/flowc.log flow-tools version 0.67: built by root@netflow on Mon Mar 21 21:21:42 EST 2005 Jonathan Glass Information Security Engineer III OIT Information Security Georgia Institute of Technology ----- End forwarded message -----
Peter Jeremy
2006-Jan-06 03:15 UTC
[jonathan.glass@oit.gatech.edu: Re: [Flow-tools] Memory leak ?]
[flow-capture process too large] On Fri, 2006-Jan-06 11:08:54 +0200, Todor Dragnev wrote:>Can someone help with this ?Help how? AFAIK, flow-control/flow-capture is not a FreeBSD port so finding someone here with knowledge of it may be difficult. If you think it's a problem with FreeBSD, you're going to need to supply more information so that we can help you.>>>I use flow-control from about 1 week. My OS is FreeBSD 6.0-RELEASE #0 >>>for AMD64. All works fine but yesterday I found this in dmesg: >>> >>>Jan 4 21:30:58 katana kernel: swap_pager_getswapspace(16): failed >>>Jan 4 21:30:58 katana kernel: swap_pager_getswapspace(12): failed >>>Jan 4 21:30:58 katana kernel: swap_pager_getswapspace(6): failed...>>>518 root 1 96 0 2648M 104M select 0:33 0.00% flow-captureThis means you've run out of swap space. The top output shows that the offending process was flow-capture. Presumably you already knew this much.>>>My starting line for flow-capture is: >>> >>>/usr/local/bin/flow-capture -p /var/run/flow-capture.pid -n 287 -N 0 -w >>>/var/log/netflows/ -S 5 /127.0.0.1/8899 >>> >>>Is that huge memory usage is memory leak or I do something wrong ?The command line means nothing to me. How big a process size were you expecting? If you kill and restart the process, how big is it initially? What libraries is it using? What does it do?>>I think the easiest way to start looking at this would be to run >>flow-capture under a memory debugger of some sort, like efence >>(Electric Fence Malloc Debugger).Have you tried this suggestion? Note that phkmalloc (the standard FreeBSD malloc) has some good debugging facilities built in - check malloc(3) for details.>I'm running flow-capture on AMD64 on Fedora Core 3 with no problems. >The only issue I run into is lack of disk space! Sometimes 50GB is not >enough!Unfortunately, Jonathan didn't say what the process size he saw was so this doesn't help much. -- Peter Jeremy