LARTC - Oct 2002 - htb limiting trouble: no overlimit or dropped packets

Hi!

I''m trying to limit the uplink bandwidth using htb but no packets are
overlimit or dropped. :-(

My router config:
* Linux kernel 2.4.20-pre10 (htb v3.7) with everything built as
  modules in QoS, iptables in use.
* tc binary from htb3.6-020525.tgz
* external interface eth0: Internet access, 512 kbit/s uplink speed
* internal interface eth1: internal LAN, 100 Mbit/s

The hosts in the internal lan should be bandwidth limited and are
masqueraded by the router.

Q: Must the versions of the kernel module and the tc utility match?

Below are my commands to create the qdiscs and classes:

tc qdisc add dev eth1 root handle 1: htb default 20
tc class add dev eth1 parent 1:  classid 1:1  htb rate 64kbit ceil 64kbit burst
5k
tc class add dev eth1 parent 1:1 classid 1:10 htb rate 64kbit ceil 64kbit burst
5k
tc class add dev eth1 parent 1:1 classid 1:20 htb rate 57kbit ceil 64kbit burst
5k
tc class add dev eth1 parent 1:1 classid 1:30 htb rate 51kbit ceil 64kbit burst
5k
tc class add dev eth1 parent 1:1 classid 1:40 htb rate 6kbit burst 0
tc qdisc add dev eth1 parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev eth1 parent 1:20 handle 20: sfq perturb 10
tc qdisc add dev eth1 parent 1:30 handle 30: sfq perturb 10
tc qdisc add dev eth1 parent 1:40 handle 40: sfq perturb 10

Notes:
* No errors from the tc utility
* All filters are scrapped for testing purposes, leaving only
  unclassified packets which go the default class (this works)
* Bandwidths are really low for testing
* There are four classes with 100%, 90%, 80% and 10% bandwidth

Next I tried an FTP upload from a machine on the internal LAN to the
internet, creating the following qdisc and class statistics:

qdisc sfq 40: quantum 1514b perturb 10sec
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc sfq 30: quantum 1514b perturb 10sec
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc sfq 20: quantum 1514b perturb 10sec
 Sent 13408020 bytes 12433 pkts (dropped 0, overlimits 0)

 qdisc sfq 10: quantum 1514b perturb 10sec
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc htb 1: r2q 10 default 20 direct_packets_stat 0
 Sent 13408020 bytes 12433 pkts (dropped 0, overlimits 13422)

 class htb 1:1 root rate 64Kbit ceil 64Kbit burst 1023b cburst 1680b
 Sent 13408020 bytes 12433 pkts (dropped 0, overlimits 0)
 rate 35bps
 lended: 1136 borrowed: 0 giants: 0
 tokens: 51202 ctokens: 116901

class htb 1:10 parent 1:1 leaf 10: prio 0 rate 64Kbit ceil 64Kbit burst
1023b cburst 1680b
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
 lended: 0 borrowed: 0 giants: 0
 tokens: 102400 ctokens: 168099

class htb 1:20 parent 1:1 leaf 20: prio 0 rate 57Kbit ceil 64Kbit burst
1671b cburst 1680b
 Sent 13408020 bytes 12433 pkts (dropped 0, overlimits 0)
 rate 35bps
 lended: 11297 borrowed: 1136 giants: 0
 tokens: 130246 ctokens: 116901

class htb 1:30 parent 1:1 leaf 30: prio 0 rate 51Kbit ceil 64Kbit burst
1664b cburst 1680b
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
 lended: 0 borrowed: 0 giants: 0
 tokens: 208941 ctokens: 168099

class htb 1:40 parent 1:1 leaf 40: prio 0 rate 6Kbit ceil 6Kbit burst
1606b cburst 1606b
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
 lended: 0 borrowed: 0 giants: 0
 tokens: 1714132 ctokens: 1714132

Well, everything goes to the default class 1:20 which is OK.
However, the traffic is *not* limited! No packets overlimit or dropped!
The FTP upload succeeded with full uplink speed (512 kbit/s).

Q: What am I missing here? Why isn''t the traffic limited by HTB?

I''ve read the LARTC Howto, the required manpages, the docs at Martin
Devera''s Homepage and had a look at the Wondershaper. Unfortunately
I''m
still unable to find the bug. :-( Any hints would be appreciated!

Thanks, Walter

PS: I''ve also tried using filters u32 and fwmark but traffic was
    never limited. I guess there is something wrong with my
    class/qdisc setup in general, right?



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Walter Haidinger wrote:
> Hi!
> 
> I''m trying to limit the uplink bandwidth using htb but no packets
are
> overlimit or dropped. :-(
> 
> My router config:
> * Linux kernel 2.4.20-pre10 (htb v3.7) with everything built as
>   modules in QoS, iptables in use.
> * tc binary from htb3.6-020525.tgz
> * external interface eth0: Internet access, 512 kbit/s uplink speed
> * internal interface eth1: internal LAN, 100 Mbit/s
> 
> The hosts in the internal lan should be bandwidth limited and are
> masqueraded by the router.
> 
> Q: Must the versions of the kernel module and the tc utility match?
> 
> Below are my commands to create the qdiscs and classes:
> 
> tc qdisc add dev eth1 root handle 1: htb default 20
> tc class add dev eth1 parent 1:  classid 1:1  htb rate 64kbit ceil 64kbit
burst 5k
> tc class add dev eth1 parent 1:1 classid 1:10 htb rate 64kbit ceil 64kbit
burst 5k
> tc class add dev eth1 parent 1:1 classid 1:20 htb rate 57kbit ceil 64kbit
burst 5k
> tc class add dev eth1 parent 1:1 classid 1:30 htb rate 51kbit ceil 64kbit
burst 5k
> tc class add dev eth1 parent 1:1 classid 1:40 htb rate 6kbit burst 0
> tc qdisc add dev eth1 parent 1:10 handle 10: sfq perturb 10
> tc qdisc add dev eth1 parent 1:20 handle 20: sfq perturb 10
> tc qdisc add dev eth1 parent 1:30 handle 30: sfq perturb 10
> tc qdisc add dev eth1 parent 1:40 handle 40: sfq perturb 10
> 
> Notes:
> * No errors from the tc utility
> * All filters are scrapped for testing purposes, leaving only
>   unclassified packets which go the default class (this works)
> * Bandwidths are really low for testing
> * There are four classes with 100%, 90%, 80% and 10% bandwidth
> 
> Next I tried an FTP upload from a machine on the internal LAN to the
> internet, creating the following qdisc and class statistics:
> 
> qdisc sfq 40: quantum 1514b perturb 10sec
>  Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> 
>  qdisc sfq 30: quantum 1514b perturb 10sec
>  Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> 
>  qdisc sfq 20: quantum 1514b perturb 10sec
>  Sent 13408020 bytes 12433 pkts (dropped 0, overlimits 0)
> 
>  qdisc sfq 10: quantum 1514b perturb 10sec
>  Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> 
>  qdisc htb 1: r2q 10 default 20 direct_packets_stat 0
>  Sent 13408020 bytes 12433 pkts (dropped 0, overlimits 13422)
> 
>  class htb 1:1 root rate 64Kbit ceil 64Kbit burst 1023b cburst 1680b
>  Sent 13408020 bytes 12433 pkts (dropped 0, overlimits 0)
>  rate 35bps
>  lended: 1136 borrowed: 0 giants: 0
>  tokens: 51202 ctokens: 116901
> 
> class htb 1:10 parent 1:1 leaf 10: prio 0 rate 64Kbit ceil 64Kbit burst
> 1023b cburst 1680b
>  Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
>  lended: 0 borrowed: 0 giants: 0
>  tokens: 102400 ctokens: 168099
> 
> class htb 1:20 parent 1:1 leaf 20: prio 0 rate 57Kbit ceil 64Kbit burst
> 1671b cburst 1680b
>  Sent 13408020 bytes 12433 pkts (dropped 0, overlimits 0)
>  rate 35bps
>  lended: 11297 borrowed: 1136 giants: 0
>  tokens: 130246 ctokens: 116901
> 
> class htb 1:30 parent 1:1 leaf 30: prio 0 rate 51Kbit ceil 64Kbit burst
> 1664b cburst 1680b
>  Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
>  lended: 0 borrowed: 0 giants: 0
>  tokens: 208941 ctokens: 168099
> 
> class htb 1:40 parent 1:1 leaf 40: prio 0 rate 6Kbit ceil 6Kbit burst
> 1606b cburst 1606b
>  Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
>  lended: 0 borrowed: 0 giants: 0
>  tokens: 1714132 ctokens: 1714132
> 
> Well, everything goes to the default class 1:20 which is OK.
> However, the traffic is *not* limited! No packets overlimit or dropped!
> The FTP upload succeeded with full uplink speed (512 kbit/s).
> 
> Q: What am I missing here? Why isn''t the traffic limited by HTB?
> 
> I''ve read the LARTC Howto, the required manpages, the docs at
Martin
> Devera''s Homepage and had a look at the Wondershaper.
Unfortunately I''m
> still unable to find the bug. :-( Any hints would be appreciated!
> 
> Thanks, Walter
> 
> PS: I''ve also tried using filters u32 and fwmark but traffic was
>     never limited. I guess there is something wrong with my
>     class/qdisc setup in general, right?
> 
> 
> 
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> 

Walter,

   I''m giving a try on this one, I''m not familiar with HTB at
all...

   I think that your only problem is: all classes can borrow bandwidth 
from other classes !!  So there is no bandwidth control !

   You should add the "bounded" modifier to classes that should be
limited.

Regards,

Stephane.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

On Thu, 17 Oct 2002, Stephane Ouellette wrote:
>    I''m giving a try on this one, I''m not familiar with
HTB at all...
Thanks, I appreciate that!
>    I think that your only problem is: all classes can borrow bandwidth
> from other classes !!  So there is no bandwidth control !
Yes, but there is a ceiling of 64 kbit/s set for each class. Yet this is
exceeded and traffic flows with full 512 kbit/s.

Besides, if you have a look at the class statistics you''ll find that
the
borrowed packet counts remain at a value of zero.
>    You should add the "bounded" modifier to classes that should
be limited.
AFAIK there is no bounded (or isolated, borrow, etc like in CBQ)
option with htb. From the htb manpage:

       tc class  ...  dev  dev  parent  major:[minor]  [  classid
       major:minor  ]  htb  rate rate [ ceil rate ] burst bytes [
       cburst bytes ] [ prio priority ]

Regards, Walter

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

On Thu, 17 Oct 2002, Nickola Kolev wrote:
> On Thu, 17 Oct 2002 00:33:05 +0200 (CEST)
> Walter Haidinger <walter.haidinger@gmx.at> wrote:
>
> [ cut ]
> > tc qdisc add dev eth1 root handle 1: htb default 20
> > tc class add dev eth1 parent 1:  classid 1:1  htb rate 64kbit ceil
64kbit burst 5k
> > tc class add dev eth1 parent 1:1 classid 1:10 htb rate 64kbit ceil
64kbit burst 5k
> > tc class add dev eth1 parent 1:1 classid 1:20 htb rate 57kbit ceil
64kbit burst 5k
> > tc class add dev eth1 parent 1:1 classid 1:30 htb rate 51kbit ceil
64kbit burst 5k
> > tc class add dev eth1 parent 1:1 classid 1:40 htb rate 6kbit burst 0
> > tc qdisc add dev eth1 parent 1:10 handle 10: sfq perturb 10
> > tc qdisc add dev eth1 parent 1:20 handle 20: sfq perturb 10
> > tc qdisc add dev eth1 parent 1:30 handle 30: sfq perturb 10
> > tc qdisc add dev eth1 parent 1:40 handle 40: sfq perturb 10
> >
> [ cut ]
>
> Can you give us the exact commands you issued? F.e. what are the filters?
Above are all commands issued. I have no filters attached because
bandwidth limiting does not even work for the default class, i.e.
unclassified traffic. I had filters assigned to the root class 1: like the
one below in an attempt to catch all packets.

tc filter add dev eth1 parent 1: protocol ip \
     prio 20 u32 match ip dst 0.0.0.0/0 flowid 1:20

Didn''t make any difference, though.

Walter


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

On Thursday 17 October 2002 00:33, Walter Haidinger
wrote:
> Hi!
>
> I''m trying to limit the uplink bandwidth using htb but no packets
are
> overlimit or dropped. :-(
>
> My router config:
> * Linux kernel 2.4.20-pre10 (htb v3.7) with everything built as
>   modules in QoS, iptables in use.
> * tc binary from htb3.6-020525.tgz
> * external interface eth0: Internet access, 512 kbit/s uplink speed
> * internal interface eth1: internal LAN, 100 Mbit/s
>
> The hosts in the internal lan should be bandwidth limited and are
> masqueraded by the router.
>
> Q: Must the versions of the kernel module and the tc utility match?
No
> Below are my commands to create the qdiscs and classes:
>
> tc qdisc add dev eth1 root handle 1: htb default 20
> tc class add dev eth1 parent 1:  classid 1:1  htb rate 64kbit ceil 64kbit
> burst 5k 
> tc class add dev eth1 parent 1:1 classid 1:10 htb rate 64kbit ceil
> 64kbit burst 5k
> tc class add dev eth1 parent 1:1 classid 1:20 htb rate 57kbit ceil 
> 64kbit burst 5k 
> tc class add dev eth1 parent 1:1 classid 1:30 htb rate 51kbit ceil 
> 64kbit burst 5k 
> tc class add dev eth1 parent 1:1 classid 1:40 htb rate 6kbit burst 0 
Htb nor cbq will be perfectly shaping.  So try ceil 57 and see what happens.  
Go as low as needed untill you can see the difference.  You have to do this.  
If you don''t, you are filling the buffers of the modem and it will be
the
modem that controls the link.
Also, make the sum of the rates from the classes equal to the rate.  If you 
have ceil 64 kbit, all classes will be able to share from each other.  So if 
one class is not using all it''s bandwidth, an other class can borrow
the
unused bandwidth.
> Well, everything goes to the default class 1:20 which is OK.
> However, the traffic is *not* limited! No packets overlimit or dropped!
> The FTP upload succeeded with full uplink speed (512 kbit/s).
> Q: What am I missing here? Why isn''t the traffic limited by HTB?
It''s limited, but you give the traffic full bandwidth because you
specified
ceil 64kbit.  Try ceil 10kbit and you will be limited at 10kbit.
> I''ve read the LARTC Howto, the required manpages, the docs at
Martin
> Devera''s Homepage and had a look at the Wondershaper.
Unfortunately I''m
> still unable to find the bug. :-( Any hints would be appreciated!
See www.docum.org for more script/info about shaping.
> PS: I''ve also tried using filters u32 and fwmark but traffic was
>     never limited. I guess there is something wrong with my
>     class/qdisc setup in general, right?
Yes :)

Stef

-- 

stef.coene@docum.org
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

On Thu, 17 Oct 2002, Stef Coene wrote:
> Htb nor cbq will be perfectly shaping.  So try ceil 57 and see what
happens.
> Go as low as needed untill you can see the difference.  You have to do
this.
> If you don''t, you are filling the buffers of the modem and it will
be the
> modem that controls the link.
I do not require perfect shaping. The problem is, that traffic doesn''t
seem to be shaped at all!
> Also, make the sum of the rates from the classes equal to the rate.  If you
> have ceil 64 kbit, all classes will be able to share from each other.  So
if
> one class is not using all it''s bandwidth, an other class can
borrow the
> unused bandwidth.
I''ll try this.
> It''s limited, but you give the traffic full bandwidth because you
specified
> ceil 64kbit.  Try ceil 10kbit and you will be limited at 10kbit.
I don''t think it is limited because upload speed _exceeds_ the ceil
rate!

Please correct me if I''m wrong with the following:
I''ve create a qdisc and class with:

tc qdisc add dev lo root handle 1: htb default 20
tc class add dev lo parent 1:  classid 1:1  htb rate 64kbit ceil 64kbit

Now, if my actual uplink speed is larger (512 kbit/s), traffic should be
shaped to the desired rate (64 kbit/s), right?

But for me upload traffic is still at full speed of 512 kbit/s.
How do I prevent this?
> See www.docum.org for more script/info about shaping.
Thanks for the reference!

Regards, Walter

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Erm you are shaping lo so it will only shape packets leaving the lo interface,
you probably want to try changing the
dev to your internet device....
>Please correct me if I''m wrong with the following:
>I''ve create a qdisc and class with:
>tc qdisc add dev lo root handle 1: htb default 20
>tc class add dev lo parent 1: classid 1:1 htb rate 64kbit ceil 64kbit
>Now, if my actual uplink speed is larger (512 kbit/s), traffic should be
>shaped to the desired rate (64 kbit/s), right?
>But for me upload traffic is still at full speed of 512 kbit/s.
>How do I prevent this?
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

On Thu, 17 Oct 2002, Robert Vale wrote:
> Erm you are shaping lo so it will only shape packets leaving the lo
interface, you probably want to try changing the
> dev to your internet device....
Yes, sorry. I was messing around with the htb ethloop utility.
Copy & paste bug.

Fortunately everything works now. See the reply to James B. Moore''s
port
in this thread.

Thanks for your post, though. :-)
>
> >Please correct me if I''m wrong with the following:
> >I''ve create a qdisc and class with:
> >tc qdisc add dev lo root handle 1: htb default 20
> >tc class add dev lo parent 1: classid 1:1 htb rate 64kbit ceil 64kbit
>
> >Now, if my actual uplink speed is larger (512 kbit/s), traffic should
be
> >shaped to the desired rate (64 kbit/s), right?
> >But for me upload traffic is still at full speed of 512 kbit/s.
> >How do I prevent this?
>
-- 
Walter Haidinger <walter.haidinger@gmx.at>
For contact information or PGP/GPG public key please refer to
http://haidinger.webhop.org/contact.html


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

On Thu, 17 Oct 2002, James B. Moore wrote:
> Not to familiar with HTB but in general it seems you have your
qdisc''s
> attached to your _internal_ interface, which should let you shape
> the traffic into your LAN; not outbound. if that''s what you intend
> then my mistake, I''ll go away now...
No, you''re definitely allowed to stay! ;-)
> You need to attache the qdisc''s to the uplink interface (eth0) and
sinca all
> the packets are Masq''d at that point, I think you need to use
firewall marks
> and the fw filter to classify the traffic from your internal hosts first.
Right! I obviously mixed up directions! How silly... <:-0

After attaching to eth0 everthing works now! Great!

Marking packets with iptables -t mangle works too! Masqueraded packets
have to be marked in the PREROUTING chain, locally generated packets in
the OUTPUT chain of the mangle table.

Traffic is limited according to the specified rates.

However, there is something I still don''t fully understand:
When do packets get dropped or are overlimit?

Thanks for the replies! A really great list!

Regards, Walter


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

On Sat, 19 Oct 2002, Joseph Watson wrote:
> I don''t see any mention of filters??  Don''t you have to
use a filter to put
> the traffic into the correct class?
Yes, if you want to distribute traffic between multiple classes. No, if
you''re only using the default class (doesn''t make much sense
but for
testing only, of course).

Quoting from my inital post:
* All filters are scrapped for testing purposes, leaving only
  unclassified packets which go the default class (this works)

My setup works now, if was using the wrong interface...

Regards, Walter

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

On Thu, 17 Oct 2002, Walter Haidinger wrote:
> Marking packets with iptables -t mangle works too! Masqueraded packets
> have to be marked in the PREROUTING chain, locally generated packets in
> the OUTPUT chain of the mangle table.
Only for the record: The above is true for locally (i.e. on the router
itself) generated data. Packets _not_ originating from the router have to
be marked in the FORWARD chain of the mangle table.

Walter

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/