CentOS - Mar 2009 - looking for some advice to monitor network usage in office

Hi all,

I've been asked by a college to setup a monitor to monitor a Windows
network, but on internet usage. They want to have detailed usage, i.e.
on a per IP / PC basis, and if possible to get stats for every
protocol, and see over a period of time what goes on.

My first though wat ntop, which does all of this, but it doesn't save
the data in a DB, so if the server reboots the stats are reset to 0. I
also can't get Cacti to give me stats per IP & per protocol (unless
someone knows how todo this).

I don't yet know the full network layout, but I have a feeling they're
using ADSL, and have a Windows Small Business server with ISA, and
possible Exchange as well. So, I'm either going to put a CentOS box
between the Windows box & ADSL router, or maybe even setup a CentOS
Vmware Virtual PC, force all the network to route via the VPS.

Does anyone have some suggestions / experience in setting up something
like this?

P.S. Please don't look at the fact that there's Windows on the
network. I use Linux for business purposes, not as a hobby, and we
also use Mac & Windows where the situation calls for it.

-- 

Kind Regards
Rudi Ahlers

On Wed, 25 Mar 2009 10:01:50 +0200
Rudi Ahlers wrote:
> I've been asked by a college to setup a monitor to monitor a Windows
> network, but on internet usage. They want to have detailed usage, i.e.
> on a per IP / PC basis, and if possible to get stats for every
> protocol, and see over a period of time what goes on.
What about privoxy and sawmill?

-- 
MELVILLE THEATRE ~ Melville Sask ~ http://www.melvilletheatre.com

Hi Rudy

2009/3/25 Rudi Ahlers <rudiahlers at gmail.com>:
> Hi all,
>
> I've been asked by a college to setup a monitor to monitor a Windows
> network, but on internet usage. They want to have detailed usage, i.e.
> on a per IP / PC basis, and if possible to get stats for every
> protocol, and see over a period of time what goes on.
>
> My first though wat ntop, which does all of this, but it doesn't save
> the data in a DB, so if the server reboots the stats are reset to 0. I
> also can't get Cacti to give me stats per IP & per protocol (unless
> someone knows how todo this).
>
> I don't yet know the full network layout, but I have a feeling
they're
> using ADSL, and have a Windows Small Business server with ISA, and
> possible Exchange as well. So, I'm either going to put a CentOS box
> between the Windows box & ADSL router, or maybe even setup a CentOS
> Vmware Virtual PC, force all the network to route via the VPS.
>
> Does anyone have some suggestions / experience in setting up something
> like this?
>
> P.S. Please don't look at the fact that there's Windows on the
> network. I use Linux for business purposes, not as a hobby, and we
> also use Mac & Windows where the situation calls for it.
>
> --
>
> Kind Regards
> Rudi Ahlers
If your firewall / border gateway is running linux, have a look at:

 http://www.networkuptime.com/tools/netflow/

You need an exporter that will export linux netflow records and
software that will collect and present the resultant data.

Regards,
  Andrew.

On Mar 25, 2009, at 4:01 AM, Rudi Ahlers <rudiahlers at gmail.com> wrote:
> Hi all,
>
> I've been asked by a college to setup a monitor to monitor a Windows
> network, but on internet usage. They want to have detailed usage, i.e.
> on a per IP / PC basis, and if possible to get stats for every
> protocol, and see over a period of time what goes on.
>
> My first though wat ntop, which does all of this, but it doesn't save
> the data in a DB, so if the server reboots the stats are reset to 0. I
> also can't get Cacti to give me stats per IP & per protocol (unless
> someone knows how todo this).
>
> I don't yet know the full network layout, but I have a feeling
they're
> using ADSL, and have a Windows Small Business server with ISA, and
> possible Exchange as well. So, I'm either going to put a CentOS box
> between the Windows box & ADSL router, or maybe even setup a CentOS
> Vmware Virtual PC, force all the network to route via the VPS.
>
> Does anyone have some suggestions / experience in setting up something
> like this?
>
> P.S. Please don't look at the fact that there's Windows on the
> network. I use Linux for business purposes, not as a hobby, and we
> also use Mac & Windows where the situation calls for I
Best way to do what your asking is to setup a proxy/firewall that all  
hosts have to pass through. That way the proxy/firewall can log all  
the activity and then you use a reporting program to report on the log  
data.

Squid can log all kinds of data, so can iptables. Couple that with  
NTLM/basic authentication on the squid host and you can put names with  
ip addresses.

The authentication can be transparent so if the user is logged on the  
network they auto-authenticate with the proxy.

-Ross

Rudi Ahlers wrote:
> Hi all,
> 
> I've been asked by a college to setup a monitor to monitor a Windows
> network, but on internet usage. They want to have detailed usage, i.e.
> on a per IP / PC basis, and if possible to get stats for every
> protocol, and see over a period of time what goes on.
> 
> My first though wat ntop, which does all of this, but it doesn't save
> the data in a DB, so if the server reboots the stats are reset to 0.
Are you sure you went through all the ntop options?  I thought it had 
ways to store and export data.  And it can both source and parse netflow 
data.
> I
> also can't get Cacti to give me stats per IP & per protocol (unless
> someone knows how todo this).
SNMP normally reports traffic per interface.  If you can get by with a 
historical total/max bandwidth report, point cacti or other SNMP tool at 
the switch ports facing the users.  Then use ntop for snapshots of 
protocol usage.  If, for example, you are trying to track down the 
source of a virus, you really only want to see current traffic patterns, 
not totals that include last week's bittorrent activity.
> I don't yet know the full network layout, but I have a feeling
they're
> using ADSL, and have a Windows Small Business server with ISA, and
> possible Exchange as well. So, I'm either going to put a CentOS box
> between the Windows box & ADSL router, or maybe even setup a CentOS
> Vmware Virtual PC, force all the network to route via the VPS.
> 
> Does anyone have some suggestions / experience in setting up something
> like this?
As long as you have a manged switch behind the internet router you 
should be able to set up a mirror (monitor) port to feed a copy to an 
interface running ntop without actually routing through the Linux box. 
Or, if the router supports it, it can send netflow records to something 
that understands them.


-- 
   Les Mikesell
     lesmikesell at gmail.com

Rudi Ahlers wrote:
> Hi all,
>
> I've been asked by a college to setup a monitor to monitor a Windows
> network, but on internet usage. They want to have detailed usage, i.e.
> on a per IP / PC basis, and if possible to get stats for every
> protocol, and see over a period of time what goes on.
>
> My first though wat ntop, which does all of this, but it doesn't save
> the data in a DB, so if the server reboots the stats are reset to 0. I
> also can't get Cacti to give me stats per IP & per protocol (unless
> someone knows how todo this).
>
> I don't yet know the full network layout, but I have a feeling
they're
> using ADSL, and have a Windows Small Business server with ISA, and
> possible Exchange as well. So, I'm either going to put a CentOS box
> between the Windows box & ADSL router, or maybe even setup a CentOS
> Vmware Virtual PC, force all the network to route via the VPS.
>
> Does anyone have some suggestions / experience in setting up something
> like this?
>
> P.S. Please don't look at the fact that there's Windows on the
> network. I use Linux for business purposes, not as a hobby, and we
> also use Mac & Windows where the situation calls for it.
>   
Just to add my .02, depending on the traffic level, you may do better 
with a pre-packaged distro like Endian which provides transparent proxy 
and reporting.  The community edition (what I'm using) sets up very 
easily and pretty much works out of the box.

For our mixed OS network of about 40 workstations, this serves very 
nicely and does pretty much what you're asking.  The only thing I did to 
the stock install was to have the logs ftp'd to me for archiving so they 
don't get rotated out of existence during the normal system rotation 
schedule. (client wants 1yr of history).

HTH,
-Ray