Hello, I''m trying to set up traffic accounting for my mixed NAT/Routing Shorewall configuration. It''s running as a VPN server, with routing to the Internet. Users connect via OpenVPN on their public IP address on eth0, and get assigned a static private rfc1918 address on the bridge interface br0. They are then one-on-one NAT''ed to their public IP address. Please find attached the ''shorewall dump'' file. Hopefully this should provide a little insight into my configuration. What I''ve tried to do is to count the bytes in all traffic from each user''s public IP address on eth0, but not had much success. I got Shorewall to log packets so I have some examples of what I want to catch, and this is what I''ve got: # user ''varka'' behind proxy to vpn server (establishing tunnel) Jan 16 00:52:45 blade038 kernel: Shorewall:net_dnat:REDIRECT:IN=eth0 OUTMAC=00:0a:e6:a6:84:9b:00:d0:97:cc:90:00:08:00 SRC=130.209.6.40 DST84.234.16.143 LE N=60 TOS=0x00 PREC=0x00 TTL=55 ID=24268 DF PROTO=TCP SPT=48322 DPT=443 WINDOW=5840 RES=0x00 SYN URGP=0 # internet to vpn user ''varka'' Jan 16 00:53:14 blade038 kernel: Shorewall:net2vpn:ACCEPT:IN=eth0 OUT=br0 PHYSOUT=tap0 SRC=82.46.150.73 DST=10.10.0.2 LEN=53 TOS=0x00 PREC=0x00 TTL=118 ID=15 1 PROTO=UDP SPT=3007 DPT=27015 LEN=33 # vpn user ''varka'' to internet Jan 16 00:53:18 blade038 kernel: Shorewall:vpn2net:ACCEPT:IN=br0 OUT=eth0 PHYSIN=tap0 SRC=10.10.0.2 DST=207.46.0.21 LEN=48 TOS=0x00 PREC=0x00 TTL=127 ID=6528 0 DF PROTO=TCP SPT=1550 DPT=1863 WINDOW=16384 RES=0x00 SYN URGP=0 Jan 16 00:53:19 blade038 kernel: Shorewall:vpn2net:ACCEPT:IN=br0 OUT=eth0 PHYSIN=tap0 SRC=10.10.0.2 DST=65.54.179.192 LEN=48 TOS=0x00 PREC=0x00 TTL=127 ID=65 296 DF PROTO=TCP SPT=1551 DPT=443 WINDOW=16384 RES=0x00 SYN URGP=0 I would ideally like to log this to four variables: user ''varka'', Tunneled traffic into server user ''varka'', tunneled traffic out of server user ''varka'', untunneled traffic into server user ''varka'', untunneled traffic out of server. However, if I cannot do this, having a simple total in/out for each user on eth0 (the bandwidth I pay for... :-) ) would be okay. In this example, user ''varka'' is 10.10.0.2. I would like to do this for all my users. This is eventually going to be incorporated into MRTG, if I ever get it working! Thank you all, Jan Mulders
On Sunday 15 January 2006 17:18, Jan Mulders wrote:> > Thank you all, >a) You appear to have an accounting rule as follows: #ACTION CHAIN SOURCE DESTINATION expl1 - eth0:84.234.16.143 - That is a nonsensical rule. It counts packets that enter the firewall on eth0 and have a source IP of 84.234.16.143. This won''t happen since 84.234.16.143 is an IP address on eth0. b) Accounting rules are applied in the filter table which means that DNAT has already been applied and SNAT has not yet been applied. So if you want to account for traffic from the internet to the client whose external IP address is 84.234.16.143, you want: #ACTION CHAIN SOURCE DESTINATION expl1 - eth0 10.10.0.2 Hope this helps. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Thank you, I have been able to implement this part of the filtering rules successfully. The Exp1 and Exp2 actually stood for ''experiment'', as until I got the packet logs I had no idea what was coming in or out from where. Will I have to set up my rules differently if I want to count data in/out for clients tunneling in on port 443 of their public IP, which is my next goal? This is the packet I need to deal with (I think): Jan 16 00:52:45 blade038 kernel: Shorewall:net_dnat:REDIRECT:IN =eth0 OUTMAC=00:0a:e6:a6:84:9b:00:d0:97:cc:90:00:08:00 SRC=130.209.6.40 DST84.234.16.143 LE N=60 TOS=0x00 PREC=0x00 TTL=55 ID=24268 DF PROTO=TCP SPT=48322 DPT=443 WINDOW=5840 RES=0x00 SYN URGP=0 Long story short, how do I account for REDIRECT''ed rules on the firewall? Are they simply connections to the destination port (1050 is the translated port, 443 is the original) on the fw, or are they something else? Do I account for: from / to / protocol / dest port eth0 / client_public_ip / tcp / 1050 ...or... eth0 / anywhere? / tcp / 1050 ...or... eth0 / client_public_ip / tcp / 443 ..or.. eth0/ fw / tcp / 1050 ..or.. eth0 / fw / tcp / 443 .......? Also - do I need to do anything special when creating the rule to tell it I don''t just want the SYN packets counted? Thanks, Jan On 16/01/06, Tom Eastep <teastep@shorewall.net> wrote:> > On Sunday 15 January 2006 17:18, Jan Mulders wrote: > > > > > Thank you all, > > > > > a) You appear to have an accounting rule as follows: > > #ACTION CHAIN SOURCE DESTINATION > expl1 - eth0:84.234.16.143 - > > That is a nonsensical rule. It counts packets that enter the firewall on > eth0 > and have a source IP of 84.234.16.143. This won''t happen since > 84.234.16.143 > is an IP address on eth0. > > b) Accounting rules are applied in the filter table which means that DNAT > has > already been applied and SNAT has not yet been applied. So if you want to > account for traffic from the internet to the client whose external IP > address > is 84.234.16.143, you want: > > #ACTION CHAIN SOURCE DESTINATION > expl1 - eth0 10.10.0.2 > > Hope this helps. > > -Tom > -- > Tom Eastep \ Nothing is foolproof to a sufficiently talented fool > Shoreline, \ http://shorewall.net > Washington USA \ teastep@shorewall.net > PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key > > >
On Monday 16 January 2006 15:26, Jan Mulders wrote:> Thank you, I have been able to implement this part of the filtering rules > successfully. > > The Exp1 and Exp2 actually stood for ''experiment'', as until I got the > packet logs I had no idea what was coming in or out from where. > > Will I have to set up my rules differently if I want to count data in/out > for clients tunneling in on port 443 of their public IP, which is my next > goal?Jan, I''m not writing your rules for you. This is not that hard. Draw yourself a box. That is your firewall. Now label some ports into and out of the box. These are your network interfaces. The SOURCE can specify the interface that the packet COMES INTO the firewall through -- it can also specify the IP address of the system who sent the packet. If the packet is created on the firewall, the source is $FW. The DEST indicates the interface that the packet LEAVES THE FIREWALL through. It can also specify the IP address that the packet is being sent to. If the packet terminates on the firewall then you can specify the IP address that it was sent to but you don''t specify a DEST interface. Again -- accounting takes place in the filter table. So DNAT (and REDIRECT) have already been applied (the destination IP address has been rewritten). SNAT/MASQUERADE has not yet been applied so if the packet came from an internal host with an RFC 1918 address, the packet still has that address as its source. Hopefully the PROTO, PORTS, and SOURCE PORT(S) are not so problematic. Just remember that if REDIRECT or DNAT rewrites the destination PORT then that has already happened when the packet goes through the accounting chain. VPN traffic from a client to the firewall enter the firewall on a real interface and are addressed to the firewall. They terminate on the firewall so you don''t want to specify an interface name in DEST. Tunneled traffic enters the fireall on a pseudo interface (tun/tap in the case of OpenVPN) -- in your case, you are using bridging so you will have to use the bridge as the SOURCE interface with the (RFC 1918) IP address assigned to that VPN client (e.g., br0:10.10.0.2). Note: The PF_KEY IPSEC implementation in the 2.6 Kernel breaks this model but that doesn''a apply to you. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key