Hi all! I''m trying to shape traffic in a dorm''s network (4 mbit symmetrical internet link, about 200 computers, heavy p2p usage). The router is a p4xeon running linux 2.6.9 with the qnet patches (http://kem.p.lodz.pl/~peter/qnet/). When I activate ip_forward I get>20% packet loss and a lot of duplicates. Any ideas? I attach my shapingscript. Thank you very much in advance, Eduardo router:~# cat htb.new #!/bin/sh IFOUT=eth0 IFIN=eth2 # cleaning tc qdisc del dev $IFOUT root &>/dev/null tc qdisc del dev $IFOUT ingress &>/dev/null # link''s capacity CEIL=500 # 1:10 interactive traffic with the highest prio (dns, ssh...) # 1:20 interactive traffic with lower prio (radios, vcn, x11...) # 1:30 bulk (http, ftp, cvs...) # 1:40 the rest (p2p mostly) tc qdisc add dev $IFOUT root handle 1: htb default 40 tc class add dev $IFOUT parent 1: classid 1:1 htb rate ${CEIL}kbps ceil ${CEIL}kbps tc class add dev $IFOUT parent 1:1 classid 1:10 htb rate 80kbps ceil 80kbps prio 0 tc class add dev $IFOUT parent 1:1 classid 1:20 htb rate 10kbps ceil 100kbps prio 1 tc class add dev $IFOUT parent 1:1 classid 1:30 htb rate 400kbps ceil ${CEIL}kbps prio 2 tc class add dev $IFOUT parent 1:1 classid 1:40 htb rate 10kbps ceil ${CEIL}kbps prio 3 tc qdisc add dev $IFOUT parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:30 handle 30: sfq perturb 10 tc qdisc add dev $IFOUT parent 1:40 handle 40: sfq perturb 10 tc filter add dev $IFOUT parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10 tc filter add dev $IFOUT parent 1:0 protocol ip prio 2 handle 2 fw classid 1:20 tc filter add dev $IFOUT parent 1:0 protocol ip prio 3 handle 3 fw classid 1:30 tc filter add dev $IFOUT parent 1:0 protocol ip prio 4 handle 4 fw classid 1:40 NF="/usr/local/sbin/iptables -t mangle -A PREROUTING " NFl7="/usr/local/sbin/iptables -t mangle -A POSTROUTING -m layer7 " /usr/local/sbin/iptables -F -t mangle # by tos $NF -p icmp -j MARK --set-mark 0x1 $NF -p icmp -j RETURN $NF -m tos --tos 0x8 -j MARK --set-mark 0x3 #scp $NF -m tos --tos 0x8 -j RETURN $NF -m tos --tos Minimize-Cost -j MARK --set-mark 0x1 $NF -m tos --tos Minimize-Cost -j RETURN $NF -m tos --tos Maximize-Throughput -j MARK --set-mark 0x4 $NF -m tos --tos Maximize-Throughput -j RETURN $NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 0x1 $NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j RETURN # by layer 7 $NFl7 --l7proto http -j MARK --set-mark 0x3 $NFl7 --l7proto http -j RETURN $NFl7 --l7proto ftp -j MARK --set-mark 0x3 $NFl7 --l7proto ftp -j RETURN $NFl7 --l7proto skype -j MARK --set-mark 0x2 $NFl7 --l7proto skype -j RETURN $NFl7 --l7proto msnmessenger -j MARK --set-mark 0x1 $NFl7 --l7proto msnmessenger -j RETURN $NFl7 --l7proto msn-filetransfer -j MARK --set-mark 0x3 $NFl7 --l7proto msn-filetransfer -j RETURN $NFl7 --l7proto jabber -j MARK --set-mark 0x1 $NFl7 --l7proto jabber -j RETURN $NFl7 --l7proto smtp -j MARK --set-mark 0x3 $NFl7 --l7proto smtp -j RETURN $NFl7 --l7proto pop3 -j MARK --set-mark 0x3 $NFl7 --l7proto pop3 -j RETURN $NFl7 --l7proto ssh -j MARK --set-mark 0x2 $NFl7 --l7proto ssh -j RETURN $NFl7 --l7proto dns -j MARK --set-mark 0x1 $NFl7 --l7proto dns -j RETURN $NFl7 --l7proto telnet -j MARK --set-mark 0x2 $NFl7 --l7proto telnet -j RETURN $NFl7 --l7proto cvs -j MARK --set-mark 0x3 $NFl7 --l7proto cvs -j RETURN $NFl7 --l7proto irc -j MARK --set-mark 0x1 $NFl7 --l7proto irc -j RETURN $NFl7 --l7proto yahoo -j MARK --set-mark 0x1 $NFl7 --l7proto yahoo -j RETURN $NFl7 --l7proto counterstrike -j MARK --set-mark 0x1 $NFl7 --l7proto counterstrike -j RETURN $NFl7 --l7proto rstp -j MARK --set-mark 0x2 $NFl7 --l7proto rstp -j RETURN $NFl7 --l7proto live365 -j MARK --set-mark 0x2 $NFl7 --l7proto live365 -j RETURN $NFl7 --l7proto h323 -j MARK --set-mark 0x2 $NFl7 --l7proto h323 -j RETURN $NFl7 --l7proto quake-halflife -j MARK --set-mark 0x1 $NFl7 --l7proto quake-halflife -j RETURN $NFl7 --l7proto rdp -j MARK --set-mark 0x2 $NFl7 --l7proto rdp -j RETURN $NFl7 --l7proto rlogin -j MARK --set-mark 0x2 $NFl7 --l7proto rlogin -j RETURN $NFl7 --l7proto shoutcast -j MARK --set-mark 0x2 $NFl7 --l7proto shoutcast -j RETURN $NFl7 --l7proto validcertssl -j MARK --set-mark 0x3 $NFl7 --l7proto validcertssl -j RETURN $NFl7 --l7proto vnc -j MARK --set-mark 0x2 $NFl7 --l7proto vnc -j RETURN $NFl7 --l7proto x11 -j MARK --set-mark 0x2 $NFl7 --l7proto x11 -j RETURN # end _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Ypu said ypu''ve got 4Mbit. As far as I''m concerned 4Mbit= 1024*4=4096 So link capacity should be 4000kbit (rate and ceil of base class) Am I wrong? ----- Original Message ----- From: "Eduardo Fernández" <eduardo@cmusanjuan.com> To: <lartc@mailman.ds9a.nl> Sent: Monday, November 15, 2004 3:52 PM Subject: [LARTC] Packet loss with htb+sfq+l7filter> Hi all! > > I''m trying to shape traffic in a dorm''s network (4 mbit symmetrical > internet link, about 200 computers, heavy p2p usage). The router is a > p4xeon running linux 2.6.9 with the qnet patches > (http://kem.p.lodz.pl/~peter/qnet/). When I activate ip_forward I get > >20% packet loss and a lot of duplicates. Any ideas? I attach my shaping > script. > > Thank you very much in advance, > > Eduardo > > router:~# cat htb.new > #!/bin/sh > IFOUT=eth0 > IFIN=eth2 > > # cleaning > tc qdisc del dev $IFOUT root &>/dev/null > tc qdisc del dev $IFOUT ingress &>/dev/null > > # link''s capacity > CEIL=500 > > # 1:10 interactive traffic with the highest prio (dns, ssh...) > # 1:20 interactive traffic with lower prio (radios, vcn, x11...) > # 1:30 bulk (http, ftp, cvs...) > # 1:40 the rest (p2p mostly) > tc qdisc add dev $IFOUT root handle 1: htb default 40 > tc class add dev $IFOUT parent 1: classid 1:1 htb rate ${CEIL}kbps ceil > ${CEIL}kbps > tc class add dev $IFOUT parent 1:1 classid 1:10 htb rate 80kbps ceil > 80kbps prio 0 > tc class add dev $IFOUT parent 1:1 classid 1:20 htb rate 10kbps ceil > 100kbps prio 1 > tc class add dev $IFOUT parent 1:1 classid 1:30 htb rate 400kbps ceil > ${CEIL}kbps prio 2 > tc class add dev $IFOUT parent 1:1 classid 1:40 htb rate 10kbps ceil > ${CEIL}kbps prio 3 > > tc qdisc add dev $IFOUT parent 1:10 handle 10: sfq perturb 10 > tc qdisc add dev $IFOUT parent 1:20 handle 20: sfq perturb 10 > tc qdisc add dev $IFOUT parent 1:30 handle 30: sfq perturb 10 > tc qdisc add dev $IFOUT parent 1:40 handle 40: sfq perturb 10 > > tc filter add dev $IFOUT parent 1:0 protocol ip prio 1 handle 1 fw > classid 1:10 > tc filter add dev $IFOUT parent 1:0 protocol ip prio 2 handle 2 fw > classid 1:20 > tc filter add dev $IFOUT parent 1:0 protocol ip prio 3 handle 3 fw > classid 1:30 > tc filter add dev $IFOUT parent 1:0 protocol ip prio 4 handle 4 fw > classid 1:40 > > NF="/usr/local/sbin/iptables -t mangle -A PREROUTING " > NFl7="/usr/local/sbin/iptables -t mangle -A POSTROUTING -m layer7 " > > /usr/local/sbin/iptables -F -t mangle > > # by tos > $NF -p icmp -j MARK --set-mark 0x1 > $NF -p icmp -j RETURN > $NF -m tos --tos 0x8 -j MARK --set-mark 0x3 #scp > $NF -m tos --tos 0x8 -j RETURN > $NF -m tos --tos Minimize-Cost -j MARK --set-mark 0x1 > $NF -m tos --tos Minimize-Cost -j RETURN > $NF -m tos --tos Maximize-Throughput -j MARK --set-mark 0x4 > $NF -m tos --tos Maximize-Throughput -j RETURN > $NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 0x1 > $NF -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j RETURN > > # by layer 7 > $NFl7 --l7proto http -j MARK --set-mark 0x3 > $NFl7 --l7proto http -j RETURN > $NFl7 --l7proto ftp -j MARK --set-mark 0x3 > $NFl7 --l7proto ftp -j RETURN > $NFl7 --l7proto skype -j MARK --set-mark 0x2 > $NFl7 --l7proto skype -j RETURN > $NFl7 --l7proto msnmessenger -j MARK --set-mark 0x1 > $NFl7 --l7proto msnmessenger -j RETURN > $NFl7 --l7proto msn-filetransfer -j MARK --set-mark 0x3 > $NFl7 --l7proto msn-filetransfer -j RETURN > $NFl7 --l7proto jabber -j MARK --set-mark 0x1 > $NFl7 --l7proto jabber -j RETURN > $NFl7 --l7proto smtp -j MARK --set-mark 0x3 > $NFl7 --l7proto smtp -j RETURN > $NFl7 --l7proto pop3 -j MARK --set-mark 0x3 > $NFl7 --l7proto pop3 -j RETURN > $NFl7 --l7proto ssh -j MARK --set-mark 0x2 > $NFl7 --l7proto ssh -j RETURN > $NFl7 --l7proto dns -j MARK --set-mark 0x1 > $NFl7 --l7proto dns -j RETURN > $NFl7 --l7proto telnet -j MARK --set-mark 0x2 > $NFl7 --l7proto telnet -j RETURN > $NFl7 --l7proto cvs -j MARK --set-mark 0x3 > $NFl7 --l7proto cvs -j RETURN > $NFl7 --l7proto irc -j MARK --set-mark 0x1 > $NFl7 --l7proto irc -j RETURN > $NFl7 --l7proto yahoo -j MARK --set-mark 0x1 > $NFl7 --l7proto yahoo -j RETURN > $NFl7 --l7proto counterstrike -j MARK --set-mark 0x1 > $NFl7 --l7proto counterstrike -j RETURN > $NFl7 --l7proto rstp -j MARK --set-mark 0x2 > $NFl7 --l7proto rstp -j RETURN > $NFl7 --l7proto live365 -j MARK --set-mark 0x2 > $NFl7 --l7proto live365 -j RETURN > $NFl7 --l7proto h323 -j MARK --set-mark 0x2 > $NFl7 --l7proto h323 -j RETURN > $NFl7 --l7proto quake-halflife -j MARK --set-mark 0x1 > $NFl7 --l7proto quake-halflife -j RETURN > $NFl7 --l7proto rdp -j MARK --set-mark 0x2 > $NFl7 --l7proto rdp -j RETURN > $NFl7 --l7proto rlogin -j MARK --set-mark 0x2 > $NFl7 --l7proto rlogin -j RETURN > $NFl7 --l7proto shoutcast -j MARK --set-mark 0x2 > $NFl7 --l7proto shoutcast -j RETURN > $NFl7 --l7proto validcertssl -j MARK --set-mark 0x3 > $NFl7 --l7proto validcertssl -j RETURN > $NFl7 --l7proto vnc -j MARK --set-mark 0x2 > $NFl7 --l7proto vnc -j RETURN > $NFl7 --l7proto x11 -j MARK --set-mark 0x2 > $NFl7 --l7proto x11 -j RETURN > # end > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Yeah, 4 mbit, that is, 512 kbps. Notice the command line:
CEIL=500
(...) rate ${CEIL}kbps ceil ${CEIL}kbps
^^^^ ^^^^
El lun, 15-11-2004 a las 23:29 +0100, mjoachimiak
escribió:> Ypu said ypu''ve got 4Mbit. As far as I''m concerned 4Mbit=
1024*4=4096
> So link capacity should be 4000kbit (rate and ceil of base class)
> Am I wrong?
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi, Eduardo Fernández wrote:> (http://kem.p.lodz.pl/~peter/qnet/). When I activate ip_forward I get > >20% packet loss and a lot of duplicates. Any ideas? I attach my shapingwell we''ve done something similar but instead of using a route we used a bridge (acting as traffic shaper and firewall). So you don''t need ip_forward maybe this is worth a try in your setup? In general a box like your should easily handle much more traffic that 4+4MBit/s. Regards, Eicke _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi Eicke, But setting up a bridge there''ll be another machine routing, so I''ll need ip_forward either way, and I won''t avoid the problem if it''s really ip_forward, I will only move it to another machine. Regards, Eduardo El lun, 15-11-2004 a las 23:37 +0100, Eicke Friedrich escribió:> Hi, > Eduardo Fernández wrote: > > (http://kem.p.lodz.pl/~peter/qnet/). When I activate ip_forward I get > > >20% packet loss and a lot of duplicates. Any ideas? I attach my shaping > well we''ve done something similar but instead of using a route we used > a bridge (acting as traffic shaper and firewall). So you don''t need > ip_forward maybe this is worth a try in your setup? In general a box > like your should easily handle much more traffic that 4+4MBit/s._______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Eduardo Fernández wrote:> Yeah, 4 mbit, that is, 512 kbps. Notice the command line: > > CEIL=500 > (...) rate ${CEIL}kbps ceil ${CEIL}kbps > ^^^^ ^^^^It''s still a bit too close to link speed - uplink allow for overheads - downlink you need to be below link speed to have a queue - its fill rate depending on the % of link used. How are you measuring packet loss? Andy. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/