Hello!
I have a quite complicated setup. In my network on each interface there is
bandwidth limitation for each user. Booth outgoing (on interface itself) and
incoming (attached IMQ) traffic. There is main HTB class which limits
bandwidth for whole interface and HTB subclasses for each user. Filtering is
done with hashing filters. This setup was working correctly.
But now in the network I have a DC hub (p2p) which allows user to exchange
files. The problem is that I want in-network p2p connections to be faster
than Internet bandwidth (and not "eating" their Internet bandwidth)
and also
I want in-network connections to have lower priority in borrowing the link
bandwidth (I am using radio links between network segments and having
"good"
Internet access is priority over local traffic).
Here is current setup:
tc qdisc del root dev eth2.24 2>/dev/null
tc qdisc add root dev eth2.24 handle 1: htb default 1
# main rate limitation for whole connection (802.11a radio link)
tc class add dev eth2.24 parent 1: classid 1:1 htb rate 15000kbit ceil
15000kbit burst 10kbit
# class for internet connections - this one can use nearly whole link
tc class add dev eth2.24 parent 1:1 classid 1:6667 htb rate 12000kbit ceil
13500kbit burst 100kbit
# class for in-network p2p connections - this one has lower guaranteed rate
tc class add dev eth2.24 parent 1:1 classid 1:6666 htb rate 3000kbit ceil
14000kbit burst 10kbit
tc qdisc add dev eth2.24 parent 1:6666 handle 6666: sfq perturb 5 quantum
1500b
# users (htb+sfq for each):
tc class add dev eth2.24 parent 1:6667 classid 1:2 htb rate 1kbit ceil 256kbit
quantum 2000 burst 100kbit
tc qdisc add dev eth2.24 parent 1:2 handle 2: sfq perturb 5 quantum 1500b
tc class add dev eth2.24 parent 1:6667 classid 1:3 htb rate 1kbit ceil 256kbit
quantum 2000 burst 100kbit
tc qdisc add dev eth2.24 parent 1:3 handle 3: sfq perturb 5 quantum 1500b
....
....
tc class add dev eth2.24 parent 1:6667 classid 1:1006 htb rate 1kbit ceil
384kbit quantum 2000 burst 100kbit
tc qdisc add dev eth2.24 parent 1:1006 handle 1006: sfq perturb 5 quantum
1500b
(some hashing filters - which I think are working properly, so not shown here)
# traffic going to these networks goes to hashing filters
tc filter add dev eth2.24 protocol ip parent 1:0 u32 match ip dst
192.168.1.0/24 hashkey mask 0x000000ff at 16 link 2:
$TC filter add dev eth2.24 protocol ip parent 1:0 u32 match ip dst
192.168.3.0/24 hashkey mask 0x000000ff at 16 link 3:
# now in-network p2p traffic. if it comes from other LANs then direct it to
# 1:6666
tc filter add dev eth2.24 protocol ip parent 1:0 u32 match ip src
192.168.5.0/24 flowid 1:6666
tc filter add dev eth2.24 protocol ip parent 1:0 u32 match ip src
192.168.4.0/24 flowid 1:6666
Filters look OK. AFAIK the last filter is the most important so even trafiic
to 192.168.1.0/24 but comming from 192.168.5.0/24 will go to 1:6666 instead
of hashing filters and user bandwidth HTB. The hashing filtes were working on
previous setup and now users still have their proper bandwidth from the
Internet.
Here is the graph representing the traffic:
http://tuxpowered.net/lan_p2p/lan_eth1_rx_dzien.png
the colors are:
#ff00ff - htb 1:6666 (LAN p2p)
#00ff00 - htb 1:6667 (Internet)
#000000 - real interface traffic (tc -s li show ethXX)
And now description of the problem:
Class 1:6666 never has more traffic than ''rate''. AFAIK it
should be having
''rate'' as minimum guaranted rate and going up to
''ceil'' if there is free
bandwidth. (directing LAN traffic to 1:1 works OK - look at the graph at
about 14:45 - 14:55)
Bandwidth sharing setup works for me on WAN interfaces, but there I have only
1 level of classes tree. Here are 2 levels and it is not working.
What I want:
.---------------------------------.
| 1:1 rate = ceil = link bandwidth|
`---------------------------------''
| |
.--------------------. .-------------------------.
| 1:6666 p2p traffic | | 1:6667 Internet traffic |
| low priority in | | high prioriy in |
| badwidth sharing: | | bandwidth sharing: |
| ceil =~ from 1:1 | | ceil =~ from 1:1 |
| rate = small | | rate = (from1:1)-1:6666 |
`--------------------'' `-------------------------''
|
+-- 1:2 user rate=1kbit ceil=256kbit
+-- 1:2 user rate=1kbit ceil=256kbit
.....
+-- 1:1203 user rate=1kbit ceil=384kbit
--
| pozdrawiam / greetings | powered by Trustix, Gentoo and FreeBSD |
| Kajetan Staszkiewicz | JID: vegeta@chrome.pl |
| Vegeta | IMQ devnames: http://tuxpowered.net |
`------------------------^----------------------------------------''
Kajetan Staszkiewicz wrote:> Here is current setup: > > tc qdisc del root dev eth2.24 2>/dev/null > tc qdisc add root dev eth2.24 handle 1: htb default 1 > > # main rate limitation for whole connection (802.11a radio link) > tc class add dev eth2.24 parent 1: classid 1:1 htb rate 15000kbit ceil > 15000kbit burst 10kbitBurst too small - it''s realated to HZ and also should be at least as big as child bursts. http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm#burst> > # class for internet connections - this one can use nearly whole link > tc class add dev eth2.24 parent 1:1 classid 1:6667 htb rate 12000kbit ceil > 13500kbit burst 100kbit > > # class for in-network p2p connections - this one has lower guaranteed rate > tc class add dev eth2.24 parent 1:1 classid 1:6666 htb rate 3000kbit ceil > 14000kbit burst 10kbit > tc qdisc add dev eth2.24 parent 1:6666 handle 6666: sfq perturb 5 quantum > 1500bI would condider using htb prio here and sfq peturb causes packet reordering so 5 is a bit low. SFQ is really best for bulk traffic.> # now in-network p2p traffic. if it comes from other LANs then direct it to > # 1:6666 > tc filter add dev eth2.24 protocol ip parent 1:0 u32 match ip src > 192.168.5.0/24 flowid 1:6666 > tc filter add dev eth2.24 protocol ip parent 1:0 u32 match ip src > 192.168.4.0/24 flowid 1:6666I think these should be before the other filters. Andy.
Dnia wtorek, 10 stycznia 2006 00:57, Andy Furniss napisał(a):> > # main rate limitation for whole connection (802.11a radio link) > > tc class add dev eth2.24 parent 1: classid 1:1 htb rate 15000kbit ceil > > 15000kbit burst 10kbit > > Burst too small - it''s realated to HZ and also should be at least as big > as child bursts. > > http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm#burstThat was the problem. Now I don''t specify burst, so tc calculates it by itself and now all is working as I wanted. Thank you. -- | pozdrawiam / greetings | powered by Trustix, Gentoo and FreeBSD | | Kajetan Staszkiewicz | JID: vegeta@chrome.pl | | Vegeta | IMQ devnames: http://tuxpowered.net | `------------------------^----------------------------------------''