Hi everybody! I''m using an edge bridge box with two ethernet cards to shape traffic in a WAN link. I''m running Debian 3.1 stable with kernel 2.6.8 and iproute from packages. I recompiled the kernel with the following built-in options: [*] 802.1d Ethernet Bridging [*] QoS and/or fair queueing <*> HTB packet scheduler <*> SFQ queue [*] QoS support <*> Firewall based classifier <*> U32 classifier I can mark packages with iptables, but I _can''t_ make the packages get queued in an specific class. Please, take a look on this sample script that guarantees 64kbit (rate), 72kbit (ceil) to the packets coming from the host 172.16.0.185 to the 172.16.1.0/24 network with 10000-10100 destination ports: # Flush tc qdiscs: /sbin/tc qdisc del dev eth0 root >/dev/null 2>&1 /sbin/tc qdisc del dev eth1 root >/dev/null 2>&1 # Create root qdiscs: /sbin/tc qdisc add dev eth0 root handle 1: htb default 1000 /sbin/tc qdisc add dev eth1 root handle 2: htb default 1000 # Create root class (1:1 2:1) : /sbin/tc class add dev eth0 parent 1: classid 1:1 htb rate 3456kbit ceil 3456kbit burst 0 prio 1 /sbin/tc class add dev eth1 parent 2: classid 2:1 htb rate 3584kbit ceil 3584kbit burst 0 prio 1 # Create default class (1:1000 2:1000) : /sbin/tc class add dev eth0 parent 1:1 classid 1:1000 htb rate 3350kbit ceil 3500kbit burst 0 prio 5 /sbin/tc class add dev eth1 parent 2:1 classid 2:1000 htb rate 3350kbit ceil 3500kbit burst 0 prio 5 # Create classes (1:44 2:44) : /sbin/tc class add dev eth0 parent 1:1 classid 1:44 htb rate 64kbit ceil 72kbit burst 8kbit prio 1 quantum 1536 /sbin/tc class add dev eth1 parent 2:1 classid 2:44 htb rate 64kbit ceil 72kbit burst 8kbit prio 1 quantum 1536 # Flushing iptables rules: /sbin/iptables -F -t mangle # iptables classify /sbin/iptables -A POSTROUTING -t mangle -p udp -s 172.16.0.185/32 --sport 10000:10100 -d 172.16.1.0/24 -j CLASSIFY --set-class 2:44 Now the statistics: bridge:~# iptables -L -t mangle -v 34302 2415K CLASSIFY udp -- any any 172.16.0.185 172.16.1.0/24 udp spts:10000:10100 CLASSIFY set 2:44 bridge:~# tc -s cl sh dev eth1 class htb 2:44 parent 2:1 prio 1 rate 64000bit ceil 72000bit burst 1023b cburst 1608b Sent 0 bytes 0 pkts (dropped 0, overlimits 0) lended: 0 borrowed: 0 giants: 0 tokens: 131070 ctokens: 183067 class htb 2:1 root rate 3584Kbit ceil 3584Kbit burst 2Kb cburst 2Kb Sent 4686617538 bytes 6922322 pkts (dropped 0, overlimits 0) rate 1936bit 2pps lended: 3691 borrowed: 0 giants: 0 tokens: 4461 ctokens: 4461 class htb 2:1000 parent 2:1 prio 5 rate 3350Kbit ceil 3500Kbit burst 2017b cburst 2Kb Sent 4686617538 bytes 6922322 pkts (dropped 0, overlimits 0) rate 1936bit 2pps lended: 6918631 borrowed: 3691 giants: 0 tokens: 4700 ctokens: 4543 As you can see, the packets are mraked by iptables but get queued through the default class. I''m getting frustated and I will appreciate all suggestions and comments. I''m using now -j CLASSIFY but I have used -j MARK and u32 tc filters with the same results. Thanks a lot for reading. Bye.
Diego Andrés Asenjo Gonzalez wrote:> Hi everybody! > > I''m using an edge bridge box with two ethernet cards to shape traffic in > a WAN link. I''m running Debian 3.1 stable with kernel 2.6.8 and iproute > from packages. I recompiled the kernel with the following built-in options: > > [*] 802.1d Ethernet Bridging > > [*] QoS and/or fair queueing > <*> HTB packet scheduler > <*> SFQ queue > > [*] QoS support > <*> Firewall based classifier > <*> U32 classifierAs I select everything I don''t know if this is enough - also I think you need to select classify in netfilter the section.> > I can mark packages with iptables, but I _can''t_ make the packages get > queued in an specific class. Please, take a look on this sample script > that guarantees 64kbit (rate), 72kbit (ceil)It''s nicer with htb to give interactive more bandwidth and higher prio than bulk. Also remember when setting rates that htb will see ip packets as ip length + 14 when used on ethX to the packets coming from> the host 172.16.0.185 to the 172.16.1.0/24 network with 10000-10100 > destination ports: > > # Flush tc qdiscs: > /sbin/tc qdisc del dev eth0 root >/dev/null 2>&1 > /sbin/tc qdisc del dev eth1 root >/dev/null 2>&1 > > # Create root qdiscs: > /sbin/tc qdisc add dev eth0 root handle 1: htb default 1000 > /sbin/tc qdisc add dev eth1 root handle 2: htb default 1000 > > # Create root class (1:1 2:1) : > /sbin/tc class add dev eth0 parent 1: classid 1:1 htb rate 3456kbit ceil > 3456kbit burst 0 prio 1 > /sbin/tc class add dev eth1 parent 2: classid 2:1 htb rate 3584kbit ceil > 3584kbit burst 0 prio 1 > > # Create default class (1:1000 2:1000) : > /sbin/tc class add dev eth0 parent 1:1 classid 1:1000 htb rate 3350kbit > ceil 3500kbit burst 0 prio 5 > /sbin/tc class add dev eth1 parent 2:1 classid 2:1000 htb rate 3350kbit > ceil 3500kbit burst 0 prio 5 > > # Create classes (1:44 2:44) : > /sbin/tc class add dev eth0 parent 1:1 classid 1:44 htb rate 64kbit ceil > 72kbit burst 8kbit prio 1 quantum 1536 > /sbin/tc class add dev eth1 parent 2:1 classid 2:44 htb rate 64kbit ceil > 72kbit burst 8kbit prio 1 quantum 1536 > > # Flushing iptables rules: > /sbin/iptables -F -t mangle > > # iptables classify > /sbin/iptables -A POSTROUTING -t mangle -p udp -s 172.16.0.185/32 > --sport 10000:10100 -d 172.16.1.0/24 -j CLASSIFY --set-class 2:44 > > Now the statistics: > > bridge:~# iptables -L -t mangle -v > 34302 2415K CLASSIFY udp -- any any 172.16.0.185 > 172.16.1.0/24 udp spts:10000:10100 CLASSIFY set 2:44 > > bridge:~# tc -s cl sh dev eth1 > class htb 2:44 parent 2:1 prio 1 rate 64000bit ceil 72000bit burst 1023b > cburst 1608b > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > lended: 0 borrowed: 0 giants: 0 > tokens: 131070 ctokens: 183067 > > class htb 2:1 root rate 3584Kbit ceil 3584Kbit burst 2Kb cburst 2Kb > Sent 4686617538 bytes 6922322 pkts (dropped 0, overlimits 0) > rate 1936bit 2pps > lended: 3691 borrowed: 0 giants: 0 > tokens: 4461 ctokens: 4461 > > class htb 2:1000 parent 2:1 prio 5 rate 3350Kbit ceil 3500Kbit burst > 2017b cburst 2Kb > Sent 4686617538 bytes 6922322 pkts (dropped 0, overlimits 0) > rate 1936bit 2pps > lended: 6918631 borrowed: 3691 giants: 0 > tokens: 4700 ctokens: 4543 > > As you can see, the packets are mraked by iptables but get queued > through the default class.You need an -o eth1 in the iptables rule for a proper count. I''m getting frustated and I will appreciate> all suggestions and comments. > > I''m using now -j CLASSIFY but I have used -j MARK and u32 tc filters > with the same results.I am not sure what else is wrong I don''t use classify, maybe check that there isn''t any decimal/hex mismatch ie. try 0x44. Andy.
Hi and thanks for replying!> > As I select everything I don''t know if this is enough - also I think > you need to select classify in netfilter the section.I also select almost everything in the netfilter section, including MARK and CLASSIFY. I simplified the script mantaining the most important part (the "outgoing" traffic). I am now using the iptables MARK way, and still did not get any shape: #!/bin/bash TC=/sbin/tc IPT=/sbin/iptables # Flush tc qdiscs: $TC qdisc del dev eth1 root >/dev/null 2>&1 # Create root qdiscs: $TC qdisc add dev eth1 root handle 2: htb default 1000 # Create root class (1:1 2:1) : $TC class add dev eth1 parent 2: classid 2:1 htb rate 3584kbit ceil 3584kbit burst 0 prio 1 # Create default class (1:1000 2:1000) : $TC class add dev eth1 parent 2:1 classid 2:1000 htb rate 3300kbit ceil 3350kbit burst 0 prio 5 # Create classes (1/2:2 tel_pereira) : $TC class add dev eth1 parent 2:1 classid 2:2 htb rate 64kbit ceil 72kbit burst 8kbit prio 1 quantum 1536 # Create filters (100/200 + 2) $TC filter add dev eth1 protocol ip parent 2:0 prio 1 handle 0x44 fw flowid 2:2 # Flushing iptables rules: $IPT -F -t mangle $IPT -A FORWARD -t mangle -p udp -s 172.16.0.185/32 -i eth1 --sport 10000:10100 -d 172.16.1.0/24 -j MARK --set-mark 0x44 The statistics remains the same: bridge:~# iptables -L -t mangle -v Chain FORWARD (policy ACCEPT 76M packets, 34G bytes) pkts bytes target prot opt in out source destination 1123 67380 MARK udp -- any any 172.16.0.185 172.16.1.0/24 udp spts:10000:10100 MARK set 0x44 There are iptables marks ... bridge:~# tc -s cl sh dev eth1 class htb 2:2 parent 2:1 prio 1 rate 64Kbit ceil 72Kbit burst 1023b cburst 1691b Sent 0 bytes 0 pkts (dropped 0, overlimits 0) lended: 0 borrowed: 0 giants: 0 tokens: 128000 ctokens: 187999 But nothing is queued through the specified class :s, there is no shaping at all. I''m going insane, ¿Could it be a bug? ¿A package error? Thanks for your help!
On Wed, Mar 01, 2006 at 02:48:18PM +0000, Andy Furniss wrote:> than bulk. Also remember when setting rates that htb will see ip packets > as ip length + 14 when used on ethXCould you elaborate on this a bit? I suppose you also meant this in an earlier message when you mentioned that the overhead was not included in the bw calculations.
Diego Andrés Asenjo Gonzalez wrote:> Hi and thanks for replying! > >> >> As I select everything I don''t know if this is enough - also I think >> you need to select classify in netfilter the section. > > > > I also select almost everything in the netfilter section, including MARK > and CLASSIFY. I simplified the script mantaining the most important part > (the "outgoing" traffic). I am now using the iptables MARK way, and > still did not get any shape:Ahh OK> $IPT -A FORWARD -t mangle -p udp -s 172.16.0.185/32 -i eth1 --sport > 10000:10100 -d 172.16.1.0/24 -j MARK --set-mark 0x44This should be -o eth1 or you should be shaping it on eth0 or if it''s for the shaping box you need to do some sort of ingress shaping/policing. Andy.
Andreas Hasenack wrote:> On Wed, Mar 01, 2006 at 02:48:18PM +0000, Andy Furniss wrote: > >>than bulk. Also remember when setting rates that htb will see ip packets >>as ip length + 14 when used on ethX > > > Could you elaborate on this a bit? > I suppose you also meant this in an earlier message when you mentioned > that the overhead was not included in the bw calculations.Maybe maybe not - There are even more overheads than 14 on eth and I may also have meant dsl overheads which you need to allow for using patches because everything gets padded out to a whole number of ATM cells. As for the IP length + 14, it''s because htb uses skb->len and on eth thats ip+14 on ppp it''s just ip length - I don''t know about other interfaces, you can use HTBs counters to test it. For eth I often see people use 10 or 100mbit as ceils without adding overheads to HTB - which you can (38-14=24 I suppose), but even then in practice you need to ceil at slightly less than 10/100mbit. Andy.
Hi again!>> $IPT -A FORWARD -t mangle -p udp -s 172.16.0.185/32 -i eth1 --sport >> 10000:10100 -d 172.16.1.0/24 -j MARK --set-mark 0x44 > > > This should be -o eth1 or you should be shaping it on eth0 or if it''s > for the shaping box you need to do some sort of ingress shaping/policing.Sorry, a typo :p. You pointed clearly the "-o eth1" in the previous message. Really, is the first time that I use -i or -o in the rule. One point is that the box is a bridge between a LAN and a router, eth0 is in the LAN and eth1 in the router. So, I thought I can ommit the interface. Anyway, I think that this is an strange situation: there are marked (maybe not "well" marked) packages, but there is no traffic. I am trying with the -o option.> > Andy. >Thanks and bye.
Diego Andrés Asenjo Gonzalez wrote:> Hi again! > >>> $IPT -A FORWARD -t mangle -p udp -s 172.16.0.185/32 -i eth1 --sport >>> 10000:10100 -d 172.16.1.0/24 -j MARK --set-mark 0x44 >> >> >> >> This should be -o eth1 or you should be shaping it on eth0 or if it''s >> for the shaping box you need to do some sort of ingress shaping/policing. > > > Sorry, a typo :p. You pointed clearly the "-o eth1" in the previous > message. Really, is the first time that I use -i or -o in the rule. One > point is that the box is a bridge between a LAN and a router, eth0 is in > the LAN and eth1 in the router. So, I thought I can ommit the interface. > > Anyway, I think that this is an strange situation: there are marked > (maybe not "well" marked) packages, but there is no traffic. I am trying > with the -o option. >It could be the bridging - I''ve never tried maybe you could look into ebtables or just use tc/u32 to do the matching. Andy.
Diego Andrés Asenjo Gonzalez ha scritto:> Sorry, a typo :p. You pointed clearly the "-o eth1" in the previous > message. Really, is the first time that I use -i or -o in the rule. One > point is that the box is a bridge between a LAN and a router, eth0 is in > the LAN and eth1 in the router. So, I thought I can ommit the interface.You have to use physdev on a bridge (-m physdev --physdev-out eth1). -- ********************************************************************** Marco Innocenti Gruppo Infrastruttura e Sicurezza CINECA phone:+39 0516171553 / fax:+39 0516132198 Via Magnanelli 6/3 e-mail: innocenti@cineca.it 40033 Casalecchio di Reno Bologna (Italia) **********************************************************************
El Thursday 02 March 2006 12:59, Andy Furniss escribió:> Diego Andrés Asenjo Gonzalez wrote: > > Hi again! > > > >>> $IPT -A FORWARD -t mangle -p udp -s 172.16.0.185/32 -i eth1 --sport > >>> 10000:10100 -d 172.16.1.0/24 -j MARK --set-mark 0x44 > >> > >> This should be -o eth1 or you should be shaping it on eth0 or if it''s > >> for the shaping box you need to do some sort of ingress > >> shaping/policing. > > > > Sorry, a typo :p. You pointed clearly the "-o eth1" in the previous > > message. Really, is the first time that I use -i or -o in the rule. One > > point is that the box is a bridge between a LAN and a router, eth0 is in > > the LAN and eth1 in the router. So, I thought I can ommit the interface. > > > > Anyway, I think that this is an strange situation: there are marked > > (maybe not "well" marked) packages, but there is no traffic. I am trying > > with the -o option. > > It could be the bridging - I''ve never tried maybe you could look into > ebtables or just use tc/u32 to do the matching.Setting the apropiates routes and proxy_arp in 1, is a way to do a bridge but at ip layer. This allows to use iptables and other ip layer tools whitouth any problem, and it''s completly transparent as the bridge. Use iptables for a bridge is certanly not a good idea. That''s why ebtables was created. -- Luciano -- Luciano
Hello! Thanks to all for your responses. m.innocenti@cineca.it wrote:>You have to use physdev on a bridge (-m physdev --physdev-out eth1). > >Yes, you are right. Taken from ebtables FAQ: * Can I use ebtables with iptables? Yes, it''s possible to use ebtables together with iptables, there are no incompatibility issues. * I''m using a 2.6 or higher kernel and my iptables rules won''t match on the bridge port devices, what''s wrong? Instead there is now an iptables match module, called |physdev|, that can be used to filter on the bridge ports. I hope physdev is going to be the solution. Bye and thanks again.