Hello,
I have some strange problems with shaping. Now I''m testing Linux as
shaping bridge. Redhat Linux 7.3, kernel plain 2.4.23 with latest HTB patch
from devik and PSCHED_CLOCK_SOURCE changed to PSCHED_CPU.
Some output is at the end of the message. What is strange that according
to interface statistics traffic is shaped more or less OK.
48431590 - 48079196 = 352394 (~275kbps)
But byte count reported by tc is lower.
39061384 - 38778938 = 282446 (~220kbps)
But when I try to look into packets with tcpdump:
# tcpdump -c 10000 | fgrep -v $ip | wc -l
tcpdump: listening on br0
5
So only 5 packets should go not through filter.
What I''m doing wrong?
Mindaugas
----------------------------------------------------------------------------
-
# for w in "qdisc" "class" "filter"; do ./tc -s -d
$w show dev eth0; echo;
done
qdisc sfq 200: limit 128p quantum 1514b flows 128/1024 perturb 10sec
Sent 59048002 bytes 942023 pkts (dropped 52103102, overlimits 0)
backlog 126p
qdisc htb 1: r2q 10 default 1 direct_packets_stat 756 ver 3.14
Sent 59211007 bytes 942778 pkts (dropped 52103102, overlimits 53038514)
backlog 126p
class htb 1:200 root leaf 200: prio 0 quantum 3276 rate 256Kbit ceil 256Kbit
burst 1926b/8 mpu 0b overhead 0b cburst 1926b/8 mpu 0b overhead 0b level 0
Sent 59049862 bytes 942054 pkts (dropped 52103123, overlimits 0)
rate 30227bps 394pps backlog 126p
lended: 941933 borrowed: 0 giants: 0
tokens: -81030 ctokens: -81030
filter parent 1: protocol ip pref 1 u32
filter parent 1: protocol ip pref 1 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800
bkt 0 flowid 1:200
match d5e2a020/fffffffc at 12
----------------------------------------------------------------------------
-
$TC qdisc del dev eth0 root
$TC qdisc del dev eth1 root
$TC qdisc add dev eth0 root handle 1: htb default 1
$TC qdisc add dev eth1 root handle 1: htb default 1
$TC filter add dev eth0 parent 1: protocol ip prio 1 u32 match u32
0x${ip_hex} 0x${netmask_hex} at 12 flowid 1:$num
$TC filter add dev eth1 parent 1: protocol ip prio 1 u32 match u32
0x${ip_hex} 0x${netmask_hex} at 16 flowid 1:$num
----------------------------------------------------------------------------
-
export E=eth0;./tc -s qdisc show dev $E;cat /proc/net/dev | fgrep $E;sleep
10;./tc -s qdisc show dev $E;cat /proc/net/dev | fgrep $E
qdisc sfq 200: limit 128p quantum 1514b perturb 10sec
Sent 38778938 bytes 621714 pkts (dropped 34927008, overlimits 0)
backlog 126p
qdisc htb 1: r2q 10 default 1 direct_packets_stat 465
Sent 38904418 bytes 622179 pkts (dropped 34927008, overlimits 35550074)
backlog 126p
eth0:1374508678 12363563 0 0 0 0 0 139
48079196 619053 37 0 0 42449 0 0
qdisc sfq 200: limit 128p quantum 1514b perturb 10sec
Sent 39061384 bytes 626322 pkts (dropped 35202131, overlimits 0)
backlog 126p
qdisc htb 1: r2q 10 default 1 direct_packets_stat 467
Sent 39187428 bytes 626789 pkts (dropped 35202131, overlimits 35829953)
backlog 126p
eth0:1382616092 12439928 0 0 0 0 0 140
48431590 623540 37 0 0 42721 0 0
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Tuesday 09 December 2003 12:24, Mindaugas Riauba wrote:> Hello, > > I have some strange problems with shaping. Now I''m testing Linux as > shaping bridge. Redhat Linux 7.3, kernel plain 2.4.23 with latest HTB patch > from devik and PSCHED_CLOCK_SOURCE changed to PSCHED_CPU. > > Some output is at the end of the message. What is strange that according > to interface statistics traffic is shaped more or less OK. > 48431590 - 48079196 = 352394 (~275kbps) > But byte count reported by tc is lower. > 39061384 - 38778938 = 282446 (~220kbps) > > But when I try to look into packets with tcpdump: > # tcpdump -c 10000 | fgrep -v $ip | wc -l > tcpdump: listening on br0 > 5 > > So only 5 packets should go not through filter. > > What I''m doing wrong?I''m not sure, but I think that the network statistics are counting every bit that''s been put on the wire. And maybe this also includes retransmits. Stef -- stef.coene@docum.org "Using Linux as bandwidth manager" http://www.docum.org/ #lartc @ irc.openprojects.net _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> > I have some strange problems with shaping. Now I''m testing Linux as > > shaping bridge. Redhat Linux 7.3, kernel plain 2.4.23 with latest HTBpatch> > from devik and PSCHED_CLOCK_SOURCE changed to PSCHED_CPU. > > > > Some output is at the end of the message. What is strange thataccording> > to interface statistics traffic is shaped more or less OK. > > 48431590 - 48079196 = 352394 (~275kbps) > > But byte count reported by tc is lower. > > 39061384 - 38778938 = 282446 (~220kbps) > > > > But when I try to look into packets with tcpdump: > > # tcpdump -c 10000 | fgrep -v $ip | wc -l > > tcpdump: listening on br0 > > 5 > > > > So only 5 packets should go not through filter. > > > > What I''m doing wrong? > I''m not sure, but I think that the network statistics are counting everybit> that''s been put on the wire. And maybe this also includes retransmits.Linux box is only bridging traffic. So I think it should not generate some additional traffic. Default class is also empty. Anyway if tc report is right then HTB has problems shaping traffic because it shows only 220kbps of traffic when configured rate is 256kbps. Such behaviour is seen only when box is heavily bashed with UDP traffic plus at least one TCP session. Mindaugas _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> > I have some strange problems with shaping. Now I''m testing Linux as > > shaping bridge. Redhat Linux 7.3, kernel plain 2.4.23 with latest HTBpatch> > from devik and PSCHED_CLOCK_SOURCE changed to PSCHED_CPU. > > What I''m doing wrong? > I''m not sure, but I think that the network statistics are counting everybit> that''s been put on the wire. And maybe this also includes retransmits.OK. I did few more tests and got following results for transmited bytes. Interface is shaped at 512kbps, tests run for 100 seconds. - tc class reports 5910888 bytes (~432kbps) - ifconfig eth1 reports 6704130 bytes (~523kbps) - tc rate reports rate 66140bps (6614000 bytes, 516kbps) As you see difference is quite essential. But if I count packets: - tc class reports 15444 packets - ifconfig eth1 reports 15504 packets Results are practically the same. Could it be that HTB somewhere does not update bytes statistics? Mindaugas _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hello,
it would be helpful to see whether it does the same if
HTB is not shaping (increase rate to 100mbit) and if it
is not bridge but router.
It is important where did u read the HTB counters. On Qdisc,
inner class or leaf ? Regarding HTB''s rate counter - it
is averaged over long time to that its result is always
approximate.
I just did quick look to code and all seems ok to me.
-------------------------------
Martin Devera aka devik
Linux kernel QoS/HTB maintainer
http://luxik.cdi.cz/~devik/
On Thu, 11 Dec 2003, Mindaugas Riauba wrote:
> > > I have some strange problems with shaping. Now I''m
testing Linux as
> > > shaping bridge. Redhat Linux 7.3, kernel plain 2.4.23 with latest
HTB
> patch
> > > from devik and PSCHED_CLOCK_SOURCE changed to PSCHED_CPU.
> > > What I''m doing wrong?
> > I''m not sure, but I think that the network statistics are
counting every
> bit
> > that''s been put on the wire. And maybe this also includes
retransmits.
>
> OK. I did few more tests and got following results for transmited bytes.
> Interface is shaped at 512kbps, tests run for 100 seconds.
>
> - tc class reports 5910888 bytes (~432kbps)
> - ifconfig eth1 reports 6704130 bytes (~523kbps)
> - tc rate reports rate 66140bps (6614000 bytes, 516kbps)
>
> As you see difference is quite essential.
>
> But if I count packets:
>
> - tc class reports 15444 packets
> - ifconfig eth1 reports 15504 packets
>
> Results are practically the same.
>
> Could it be that HTB somewhere does not update bytes statistics?
>
> Mindaugas
>
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> it would be helpful to see whether it does the same if > HTB is not shaping (increase rate to 100mbit) and if it > is not bridge but router.When HTB is not shaping (0 dropped packets) difference is smaller but still considerable. ifconfig - 63510398 bytes tc - 61854134 bytes Packet rates are much closer but what is strange - tc value is higher. ifconfig - 532493 packets tc - 536796 packets And if I''ll be able I''ll test the same configuration as router. But not today.> It is important where did u read the HTB counters. On Qdisc, > inner class or leaf ? Regarding HTB''s rate counter - it > is averaged over long time to that its result is always > approximate.Counters were read from SFQ qdisc. But HTB class counters difference is practically the same. Mindaugas _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
inner subclass (SFQ) probably WILL have smaller counts when shaping.> > inner class or leaf ? Regarding HTB''s rate counter - it > > is averaged over long time to that its result is always > > approximate. > > Counters were read from SFQ qdisc. But HTB class counters > difference is practically the same. > > Mindaugas > > >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> inner subclass (SFQ) probably WILL have smaller counts > when shaping. > > > > inner class or leaf ? Regarding HTB''s rate counter - it > > > is averaged over long time to that its result is always > > > approximate. > > > > Counters were read from SFQ qdisc. But HTB class counters > > difference is practically the same.So where to get _correct_ counters? Inner subclass (SFQ) and leaf HTB shows almost the same (<1% difference) values. Mindaugas _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Depends WHAT do you want to know. All should be correct but each shows different things. In bridging scenario there may be other issues (I didn''t tested it). Counter on upper qdisc shows packets received and leaf counts when transmitting. ifconfig counts reallu send packets - it may be smaller if NIC can''t send packet (error). If you think there is bug then test with routing setup first, I have no time to test with bridge. devik On Thu, 11 Dec 2003, Mindaugas Riauba wrote:> > > > inner subclass (SFQ) probably WILL have smaller counts > > when shaping. > > > > > > inner class or leaf ? Regarding HTB''s rate counter - it > > > > is averaged over long time to that its result is always > > > > approximate. > > > > > > Counters were read from SFQ qdisc. But HTB class counters > > > difference is practically the same. > > So where to get _correct_ counters? Inner subclass (SFQ) > and leaf HTB shows almost the same (<1% difference) values. > > Mindaugas > > >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Wednesday 10 December 2003 10:57, Ben Efros wrote:> The answer is in your own email > > > > Patch against > > > 2.4.23 and 2.6.0-test10 is hereThank you but I was asking for a good reason: The only significant change in the patch itself is commented as follows: + /* This line used to be after htb_destroy_class call below + and surprisingly it worked in 2.4. But it must precede it + because filter need its target class alive to be able to call + unbind_filter on it (without Oops). */ + htb_destroy_filters(&q->filter_list); Therefore I wanted to know whether the Ooopses fixed by the patch are known to be possible with 2.4 as well or not. I am currently trying to find the cause for very rare hangups (about once in two months) on a production server which heavily uses htb. Anyone with first-hand experience? Best regards, Andreas _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> Depends WHAT do you want to know. All should be correct but > each shows different things.I want to know number of bytes sent by specific customer. Customer corresponds to SFQ leaf.> In bridging scenario there may be other issues (I didn''t > tested it). > Counter on upper qdisc shows packets received and leaf > counts when transmitting. > ifconfig counts reallu send packets - it may be smaller > if NIC can''t send packet (error). > If you think there is bug then test with routing setup first, > I have no time to test with bridge.Tested with routed setup - the same problem. When there are only TCP or only UDP data streams everything wents fine. But if I use one TCP and one UDP stream - counters start to differ. If I add 3rd TCP stream - difference between ifconfig and tc counters increases. If 3rd stream is UDP - difference decreases. If you think that taht could help - I think I can provide root access to this box. Mindaugas _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/