Shorewall users - Jul 2002 - Small HOWTO: Setting up an IP accounting system with Shorewall

--=-1enY/hOv/KihZ4lgYBlt
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

Hi All

I came to work on a Monday morning (gotta love Mondays...) with a nice
e-mail from my boss asking me to setup an IP Accounting system so he can
find out how much inbound/outbound data each workstation does on an
hourly/daily/monthly basis. We only get so many GB''s per month of
inbound data free so this information would help him find who''s ass to
kick. :) Anyway, I decided to write this little HOWTO on how I did it.
The distribution I used was Gentoo Linux v1.2 (http://www.gentoo.org)
and Shorewall v1.3.3.

The following pre-requisites had to be met:

1. Management must be able to see IP accounting statistics
(web/email/whatever)
2. Stats must be updated frequently (every few minutes if possible)
3. Stats must be accounted for on a per-workstation basis (accounting
for the entire subnet is not suffice)
4. No machine on the network is to have Internet access unless they are
part of the accounting system

In order for management to be able to keep track of stats, I thought the
best way of doing this was to provide the stats over http. ipac-ng was
an option, but it looked difficult to implement with shorewall.
Another package recommended to me was BW-ACCT
(http://www.freshmeat.net/projects/bw-acct).
The screenshot looked good, so I installed MySQL and followed the
BW-ACCT INSTALL file instructions.

It was time to setup the firewall rules to account for traffic.

Also, FWIW, in my setup: eth0 =3D external, eth1 =3D DMZ, eth2 =3D Internal.

I created a new file, /etc/shorewall/ipacct, with the following script
in it:

---------------------- cut ----------------------
#################################
#        IP Accounting          #
#################################
#
# Create chains
createchain ipacct no

in=3D"-i"
out=3D"-o"

# Files containing subnets to account for
subnets=3D`find_file ipacct.subnets`

# Inbound
#
# Internet to LAN
run_iptables -I net2loc -j ipacct
# Internet to DMZ
run_iptables -I net2dmz -j ipacct
# Internet to FW
run_iptables -I net2fw -j ipacct

# Outbound
#
# LAN to Internet
run_iptables -I loc2net -j ipacct
# DMZ to Internet
run_iptables -I dmz2net -j ipacct
# FW to Internet
run_iptables -I fw2net -j ipacct


#########
# Rules #
#########

# The following inserts rules into iptables to account
# the total inbound/outbound data for each subnet
#
# the file "/etc/shorewall/ipacct.subnets" contains
# each subnet, one per line. ie:
#
# 192.168.1.0/24
# 203.3.3.3/24
#
for sn in `cat $subnets`; do
        # Create the chain for each subnet
        iptables -N "$sn"
        # Add inbound/outbound rules to each chain
        iptables -A $sn ${in} eth0 -d $sn
        iptables -A $sn ${out} eth0 -s $sn
        # Add rules into the ipacct chain to link to each subnets chain
        iptables -A ipacct ${in} eth0 -d $sn -j $sn
        iptables -A ipacct ${out} eth0 -s $sn -j $sn
done

# Hosts
masqfile=3D`find_file masq`

if [ -f $masqfile ]; then
        accthosts=3D`cat $masqfile|egrep -v "^#|^$"|awk -F''
'' ''{print
$2}''`
        for accthost in $accthosts; do
                # Create the chain for each subnet
                iptables -N "$accthost"
                # Add inbound/outbound rules to each chain
                iptables -A $accthost ${in} eth0 -d $accthost
                iptables -A $accthost ${out} eth0 -s $accthost
                # Add rules into the ipacct chain to link to each
subnets chain
                iptables -A ipacct ${in} eth0 -d $accthost -j $accthost
                iptables -A ipacct ${out} eth0 -s $accthost -j $accthost
        done
fi

#
# Extra Hosts to account for (space separated) not listed=20
# in the /etc/shorewall/masq file
#
extrahosts=3D"192.168.1.23"
for accthost in $extrahosts; do
        # Create the chain for each subnet
        iptables -N "$accthost"
        # Add inbound/outbound rules to each chain
        iptables -A $accthost ${in} eth0 -d $accthost
        iptables -A $accthost ${out} eth0 -s $accthost
        # Add rules into the ipacct chain to link to each subnets chain
        iptables -A ipacct ${in} eth0 -d $accthost -j $accthost
        iptables -A ipacct ${out} eth0 -s $accthost -j $accthost
done

# DMZ Hosts

dmzhosts=3D"203.3.3.3 203.3.3.4 203.3.3.5"

for dmzhost in $dmzhosts; do
        # Create the chain for each subnet
        iptables -N "$dmzhost"
        # Add inbound/outbound rules to each chain
        iptables -A $dmzhost ${in} eth0 -d $dmzhost
        iptables -A $dmzhost ${out} eth0 -s $dmzhost
        # Add rules into the ipacct chain to link to each subnets chain
        iptables -A ipacct ${in} eth0 -d $dmzhost -j $dmzhost
        iptables -A ipacct ${out} eth0 -s $dmzhost -j $dmzhost
done
---------------------- cut ----------------------

So, basically, in my setup I list specifically which hosts are to be
masqueraded in /etc/shorewall/masq. This means that if I add a new
workstation, I just add it to the /etc/shorewall/masq file, restart
shorewall, add the workstation to the admin interface for BWACCT and
that''s about it. I guess I could modify BWACCT to do this for me when I
add a new host but I haven''t gotten that far yet :-)

Finally, I made the following changes to the /etc/init.d/shorewall
script to call the ''ipacct'' script above:

---------------------- cut ----------------------
+########################################################################
+# IP Accounting                                                       #
+#######################################################################
+
+add_ipacct_rules() {
+       ipacctfile=3D`find_file ipacct`
+       if [ -f $ipacctfile ]; then
+               . $ipacctfile
+       fi
+}
+
+write_ipacct_stats() {
+       echo "Writing IP Accounting stats ..."
+       /home/httpd/htdocs/iptables-stats.pl 2>&1 >/dev/null
+}
+

+    echo "Adding IP Accounting rules ..."
+
+    add_ipacct_rules


     restart)
+       write_ipacct_stats

---------------------- cut ----------------------

The "+" indicates a line added by me. Basically during the
start/restart
of shorewall (under the define_firewall section) it just adds the IP
Accounting rules (add_ipacct_rules). Under the restart section, before
it wipes all the rules & chains it runs the write_ipacct_stats section
to write the stats to the MySQL database before clearing them.

Anyway, I hope this helps someone. Tom, thank you for your great work!
:-)

Regards,

Gonzalo

--=-1enY/hOv/KihZ4lgYBlt
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQA9O/nuvTbSWKMx8O4RArfgAKCPfQieZJW72pLEQ1p5ZmXHV6NbaQCfZhmr
nNY+iWvrJCQYYpF+Nol1NHU=uCA9
-----END PGP SIGNATURE-----

--=-1enY/hOv/KihZ4lgYBlt--

On 22 Jul 2002, Gonzalo Servat wrote:
> 
> Finally, I made the following changes to the /etc/init.d/shorewall
> script to call the ''ipacct'' script above:
> 
Why didn''t you simple create /etc/shorewall/start and add your
additional
functions there?

-Tom
-- 
Tom Eastep    \ Shorewall - iptables made easy
AIM: tmeastep  \ http://www.shorewall.net
ICQ: #60745924  \ teastep@shorewall.net

On Mon, 22 Jul 2002, Tom Eastep wrote:
> On 22 Jul 2002, Gonzalo Servat wrote:
> 
> > 
> > Finally, I made the following changes to the /etc/init.d/shorewall
> > script to call the ''ipacct'' script above:
> > 
> 
> Why didn''t you simple create /etc/shorewall/start and add your
additional
> functions there?
> 
And why didn''t I proof-read?  sed s/simple/simply/

-Tom
-- 
Tom Eastep    \ Shorewall - iptables made easy
AIM: tmeastep  \ http://www.shorewall.net
ICQ: #60745924  \ teastep@shorewall.net

On Mon, 22 Jul 2002 05:52:42 -0700 (PDT) Tom Eastep
<teastep@shorewall.net> wrote:
> > Finally, I made the following changes to the /etc/init.d/shorewall
> > script to call the ''ipacct'' script above:
> 
> Why didn''t you simply create /etc/shorewall/start and add your
additional
> functions there?
Probably he wants ipacct stuff included in shorewall by default?;-)

Regards,
Nerijus

>ipac-ng was
>an option, but it looked difficult to implement with shorewall."
Why? Just run fetchipac -S everytime you modify the shorewall rules, or just
hook it into the hooks of shorewall, works very nicely indeed.

At 7/22/2002 09:26 AM, Gonzalo Servat wrote:
>I came to work on a Monday morning (gotta love Mondays...) with a nice
>e-mail from my boss asking me to setup an IP Accounting system so he can
>find out how much inbound/outbound data each workstation does on an
>hourly/daily/monthly basis.
I''ve been looking these days for something correlated to this. I found 
these two interesting open source tools:

IPBand (http://ipband.sourceforge.net/) is a pcap based IP traffic monitor. 
It tallies per-subnet traffic and bandwidth usage and starts detailed 
logging if specified threshold for the specific subnet is exceeded.

Argus (http://www.qosient.com/argus/) is a network Audit Record Generation 
and Utilization System. Argus provides a common data format for reporting 
flow metrics such as connectivity, capacity, demand, loss, delay, and 
jitter on a per transaction basis. Imagine a huge
''tcpdump-like'' database
(off-line or real-time) with a full filtering capabilities to extract any 
kind of report based on that traffic.

Just FYI.

Sorry for off-topic.

-Gilson