Hi.
I''m going to setup policy routing and traffic shaping. Before that I
need to do some kind of study about our traffic to determine the best
way of distribute our bandwidth. For that reason I''d like some tool
that could retrieve this information:
- traffic based on app layer protocol (for instance, http, ftp, pop3;
or basing on different destination ports: 80, 21, 110).
- the measurements should be statistics, I mean, I could examine
traffic from 9h to 14h and that program should give traffic per hour,
etc.
- differentiate inbound / outbound traffic
The idea is that I could determine for example if my outbound
bandwidth is abused for one or another protocol, etc...
Which tool would you use? Thanks in advance.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ** RoMaN SoFt / LLFB **
roman@madrid.com
http://pagina.de/romansoft
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bert hubert writes:> > Target[traffic]: `/var/www/mrtg.ds9a.nl/traffic` > > and /var/www/mrtg.ds9a.nl/traffic: > > #!/bin/sh > /sbin/iptables -L -n -v -x | grep mrtg- |grep "^ " | awk "{print \$2}" > uptime > uname -a >I made a little proggie to avoid piping awk or perl or any other reporting languge... Right now it uses this format: rulenum [packets:bytes] this format is for my own application, but you can modify the printf to report whatever you need with whatever separator you want, or even add another param to accept the rule number and get statistics of the desired rule. This can be adapted to mrtg so it will run faster without perl invocation. I attach the code. Nikolai
On Sat, Sep 22, 2001 at 04:41:57PM +0200, RoMaN SoFt / LLFB wrote:> > Hi. > > I''m going to setup policy routing and traffic shaping. Before that I > need to do some kind of study about our traffic to determine the best > way of distribute our bandwidth. For that reason I''d like some tool > that could retrieve this information: > - traffic based on app layer protocol (for instance, http, ftp, pop3; > or basing on different destination ports: 80, 21, 110). > - the measurements should be statistics, I mean, I could examine > traffic from 9h to 14h and that program should give traffic per hour, > etc. > - differentiate inbound / outbound trafficTricky. For instantaneous measurements, I use iptraf. Then there is ipac, which you can use for per ip/per port measurements, which can report on time intervals. I also use iptables & mrtg together like this: iptables -N mrtg-from-the-internet > /dev/null 2> /dev/null iptables -N mrtg-to-the-internet > /dev/null 2> /dev/null iptables -A INPUT -i eth0 \! -s 213.244.168.192/26 -j mrtg-from-the-internet iptables -A OUTPUT -o eth0 \! -d 213.244.168.192/26 -j mrtg-to-the-internet and then: Target[traffic]: `/var/www/mrtg.ds9a.nl/traffic` and /var/www/mrtg.ds9a.nl/traffic: #!/bin/sh /sbin/iptables -L -n -v -x | grep mrtg- |grep "^ " | awk "{print \$2}" uptime uname -a Regards, bert -- http://www.PowerDNS.com Versatile DNS Software & Services Trilab The Technology People Netherlabs BV / Rent-a-Nerd.nl - Nerd Available - ''SYN! .. SYN|ACK! .. ACK!'' - the mating call of the internet
RoMaN SoFt / LLFB <roman@madrid.com> writes:> Before that I > need to do some kind of study about our traffic to determine the best > way of distribute our bandwidth. For that reason I''d like some tool > that could retrieve this information:...> Which tool would you use? Thanks in advance.Try ntop - www.ntop.org. -- Arkadiusz Miśkiewicz IPv6 ready PLD Linux at http://www.pld.org.pl misiek(at)pld.org.pl AM2-6BONE, 1024/3DB19BBD, arekm(at)ircnet, PWr
Hello Arkadiusz,>> Before that I >> need to do some kind of study about our traffic to determine the best >> way of distribute our bandwidth. For that reason I''d like some tool >> that could retrieve this information:I use wipl, it show screen as this: Start time: 2001-09-22 22:25:35 End time : 2001-09-22 22:25:47 MAC | IP | Send | Rcvd --------------------------------------------------------------- 00:00:f0:64:04:23 | 10.0.1.17 | 0.1KB/s | 3.2KB/s 00:00:f0:64:07:25 | 10.0.1.21 | 8.1KB/s | 0.2KB/s 00:00:f0:64:07:3a | 10.0.1.16 | 0.0KB/s | 0.0KB/s 00:00:f0:64:07:98 | 10.0.1.24 | 0.2KB/s | 4.1KB/s 00:00:f0:64:08:29 | 10.0.1.42 | 0.0KB/s | 0.0KB/s 00:00:f0:64:08:4f | 10.0.1.26 | 0.2KB/s | 2.0KB/s 00:00:f0:64:08:58 | 10.0.1.12 | 0.1KB/s | 0.2KB/s 00:00:f0:64:0c:fd | 10.0.1.44 | 0.0KB/s | 0.0KB/s 00:00:f0:64:0d:00 | 10.0.1.62 | 0.2KB/s | 0.2KB/s 00:00:f0:64:0d:04 | 10.0.1.18 | 5.3KB/s | 0.6KB/s 00:00:f0:64:0d:0b | 10.0.1.22 | 0.0KB/s | 0.1KB/s 00:02:2d:29:f1:bc | 10.0.2.12 | 0.1KB/s | 3.1KB/s 00:40:96:44:58:64 | 10.0.1.34 | 9.7KB/s | 0.2KB/s 00:40:96:47:22:0b | 10.0.1.10 | 0.1KB/s | 3.6KB/s 00:50:fc:1e:3d:be | 10.0.2.50 | 0.2KB/s | 3.6KB/s 00:e0:7d:9f:3b:c8 | 10.0.1.77 | 0.1KB/s | 0.1KB/s --------------------------------------------------------------- | | 24.4KB/s | 21.2KB/s search wipl in freshmeat. Regards Fabian
hi,
first, i''d like to say im so happy this list is finally back up!!!!
my situation is that i have two cable modems connected to two different
2.4.2 boxes.
box A (eth0 x.x.x.x, eth1 10.1.1.1/24)
box B (eth0 x.x.x.x, eth2 10.2.2.1/24)
those two connect to a central 2.4.2 machine with 5 nics.
box C (eth0 10.0.0.1/24, eth1 10.1.1.2/24, eth2 10.2.2.2/24, eth3
10.3.3.1/24, eth4 10.4.4.1/24)
10.0.0.0/24 is cisco management subnet
10.1.1.0/24 is only for the box A - box C connection
10.2.2.0/24 is only for the box B - box C connection
10.3.3.0/24 is public MS lan
10.4.4.0/24 is my private lan for my linux servers and personal machines
i masquerade 10.3.3.0/24 and 10.4.4.0/24 at box C, 10.1.1.0/24 at box A,
and 10.2.2.0/24 at box B.
box C will send any outgoing masquerades based upon its default gateway.
the cable modems both fail serveral hours a week.
when when goes down, i manually switch box C to the other gateway.
when they are both up, only one is used.
obviously, you can see the problem.
the most important thing is to load balence between the two cable modems
and route all traffic out the right way if one of the cable modems goes
down.
if possible, possibly even send some packets out one way and some out
aNother (based on payload content and packet tagging?) but this is for my
next lesson :)
here are my configs:
(please feel free to point out any errors or possible enhancements to these
configs, since i am just starting out with iptables -- and linux routing in
general --- ...any suggestions would be appreciated, thanX)
----------------------------------------------------------------------------
----------------------------------------------------------------------------
---------
BOX C:
[root@io /root]# cat masquerade
#!/bin/sh
modprobe ip_tables
modprobe ip_nat_ftp
modprobe ip_conntrack
modprobe ip_conntrack_ftp
modprobe iptable_nat
echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/iptables -F
/sbin/iptables -P INPUT DROP
/sbin/iptables -A INPUT -p tcp --dport 22 -i eth+ -j ACCEPT
/sbin/iptables -A INPUT -p tcp --dport 53 -i eth3 -j ACCEPT
/sbin/iptables -A INPUT -p tcp --dport 53 -i eth4 -j ACCEPT
/sbin/iptables -A INPUT -i eth+ -m state --state ESTABLISHED,RELATED -j
ACCEPT
/sbin/iptables -A INPUT -i eth+ -p icmp -j ACCEPT
/sbin/iptables -A INPUT -i lo -j ACCEPT
/sbin/iptables -A INPUT -i eth+ -m state --state NEW,INVALID -j
LOG --log-level debug --log-prefix "FIREWALL: Input: "
/sbin/iptables -P FORWARD DROP
/sbin/iptables -A FORWARD -o eth0 -j ACCEPT
/sbin/iptables -A FORWARD -o eth1 -j ACCEPT
/sbin/iptables -A FORWARD -o eth2 -j ACCEPT
/sbin/iptables -A FORWARD -i eth3 -j ACCEPT
/sbin/iptables -A FORWARD -i eth4 -j ACCEPT
/sbin/iptables -A FORWARD -i eth+ -m state --state ESTABLISHED,RELATED -j
ACCEPT
/sbin/iptables -A FORWARD -o eth+ -j LOG --log-level debug --log-prefix
"FIREWALL: Forward-Out: "
/sbin/iptables -A FORWARD -i eth+ -j LOG --log-level debug --log-prefix
"FIREWALL: Forward-In: "
/sbin/iptables -P OUTPUT ACCEPT
/sbin/iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE
----------------------------------------------------------------------------
-----------------------------------------------------------------
BOX A & B:
[root@matador /root]# cat masquerade
#!/bin/sh
modprobe ip_tables
modprobe ip_nat_ftp
modprobe ip_conntrack
modprobe ip_conntrack_ftp
modprobe iptable_nat
echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/iptables -F
/sbin/iptables -P INPUT DROP
/sbin/iptables -A INPUT -p tcp --dport 22 -i eth+ -j ACCEPT
/sbin/iptables -A INPUT -i eth+ -m state --state ESTABLISHED,RELATED -j
ACCEPT
/sbin/iptables -A INPUT -i eth1 -p icmp -j ACCEPT
/sbin/iptables -A INPUT -i lo -j ACCEPT
/sbin/iptables -A INPUT -i eth0 -j LOG --log-level debug --log-prefix
"FIREWALL: Input: "
/sbin/iptables -P FORWARD DROP
/sbin/iptables -A FORWARD -o eth0 -j ACCEPT
/sbin/iptables -A FORWARD -i eth0 -m state --state ESTABLISHED,RELATED -j
ACCEPT
/sbin/iptables -A FORWARD -i eth1 -j ACCEPT
/sbin/iptables -A FORWARD -i eth0 -j LOG --log-level debug --log-prefix
"FIREWALL: For-In: Spoof: "
/sbin/iptables -A FORWARD -o eth1 -j LOG --log-level debug --log-prefix
"FIREWALL: For-Out: Spoof: "
/sbin/iptables -P OUTPUT ACCEPT
/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
----------------------------------------------------------------------------
--------------------------------------------------------------------------
thanks,
Fernando Pando
On Sat, Sep 22, 2001 at 05:45:50PM -0400, Largo Hellenz wrote:> hi, > > first, i''d like to say im so happy this list is finally back up!!!!I''m happy too :-)> those two connect to a central 2.4.2 machine with 5 nics. > box C (eth0 10.0.0.1/24, eth1 10.1.1.2/24, eth2 10.2.2.2/24, eth3 > 10.3.3.1/24, eth4 10.4.4.1/24)Very impressive.> the most important thing is to load balence between the two cable modems > and route all traffic out the right way if one of the cable modems goes > down.This depends greatly on what''s on the other side of the cable modems - are there two separate ISPs? This situation is very difficult to resolve properly, the best way is to experiment a bit. You will most probably need a cronscript to detect which modems are operating.> if possible, possibly even send some packets out one way and some out > aNother (based on payload content and packet tagging?) but this is for my > next lesson :)Policy routing does this for you, and may in fact be the best solution. Route part of your customers to one modem, and others to the other, if both are functioning. If you detect that stuff is down, route everybody to the other one.> [root@io /root]# cat masquerade > #!/bin/sh > > modprobe ip_tables > modprobe ip_nat_ftp > modprobe ip_conntrack > modprobe ip_conntrack_ftp > modprobe iptable_natHaving modules autoload themselves is way easier, bt.> echo 1 > /proc/sys/net/ipv4/ip_forward > > /sbin/iptables -FI would advise to change the path, so you can leave out the redundant /sbin on every line. Regards, bert -- http://www.PowerDNS.com Versatile DNS Software & Services Trilab The Technology People Netherlabs BV / Rent-a-Nerd.nl - Nerd Available - ''SYN! .. SYN|ACK! .. ACK!'' - the mating call of the internet
>-----Original Message----- >From: lartc-admin@mailman.ds9a.nl [mailto:lartc-admin@mailman.ds9a.nl]On >Behalf Of bert hubert >Sent: Sunday, September 23, 2001 10:59 AM >To: lartc@mailman.ds9a.nl >Subject: Re: [LARTC] 2 gateways ou>> the most important thing is to load balence between the two cable modems >> and route all traffic out the right way if one of the cable modems goes >> down.>This depends greatly on what''s on the other side of the cable modems - are >there two separate ISPs?yes. two different providers.>This situation is very difficult to resolve properly, the best way is to >experiment a bit. You will most probably need a cronscript to detect which >modems are operating.so i would ping both of my two gateways via the cronscripts and, if the current default gateway is down, and the standby is up, then change default gateway? but what if both are up? is this where policy routing steps in? would policy routing replace any pinging cronscripts?>Policy routing does this for you, and may in fact be the best solution. >Route part of your customers to one modem, and others to the other, if both >are functioning. If you detect that stuff is down, route everybody to the >other one.so policy routing dynamically routes based upon available paths on the fly? could anyone provide an example that just cuts traffic equally between the two? lets say someone was was browsing cars.com and it was 10 hops from one ISP but only 5 from the other.... can policy routing tell those packets to use the shorter path? or is this something that routing daemons are for? or is there another way to solve this puzzle?>> [root@io /root]# cat masquerade >> #!/bin/sh >> >> modprobe ip_tables >> modprobe ip_nat_ftp >> modprobe ip_conntrack >> modprobe ip_conntrack_ftp >> modprobe iptable_nat>Having modules autoload themselves is way easier, bt.do you mean droping these commands in an init script so they always come up at boot time? or something else? thanks, Fernando Pando
On Sun, Sep 23, 2001 at 03:04:52PM -0400, Largo Hellenz wrote:> >This depends greatly on what''s on the other side of the cable modems - are > >there two separate ISPs? > > yes. two different providers.Ok. This rules out a lot of fancy tricks.> so i would ping both of my two gateways via the cronscripts and, if the > current > default gateway is down, and the standby is up, then change default gateway?With policy routing, you can have multiple default gateways.> so policy routing dynamically routes based upon available paths on the fly? > could anyone provide an example that just cuts traffic equally between the > two?Well, you could try something like this: # ip route add default nexthop via 10.0.0.1 dev eth0 \ nexthop via 10.0.0.202 dev eth1 Not sure if this is ''sticky''. This might just assign TCP sessions to a single cablemodem, then again, it might not. I would vote against this. Your users would also fall victim to the ''AOL Proxy Problem''. In the midst of browsing a site, they continually appear from two different IP addresses, which may upset loadbalancing tools.> lets say someone was was browsing cars.com and it was 10 hops from one ISP > but only 5 from the other.... can policy routing tell those packets to use > the shorter path? or is this something that routing daemons are for? > or is there another way to solve this puzzle?You could try tricks with squid, which has been known to have some kind of support for this. Your best best is to have multiple route tables, and route these tables differently. This page appears to be relevant: http://mlarchive.ima.com/linux-net/1999/3495.html> do you mean droping these commands in an init script so they always come up > at boot time? or something else?These modules can be autoloaded by the kernel. Make sure that you have modutils 2.4.x! Regards, bert -- http://www.PowerDNS.com Versatile DNS Software & Services Trilab The Technology People Netherlabs BV / Rent-a-Nerd.nl - Nerd Available - ''SYN! .. SYN|ACK! .. ACK!'' - the mating call of the internet