LARTC - Dec 2001 - FW: Traffic shaping in conjunction with QoS

Hey guys, very new at this stuff, but getting extremely interested in the
topic.  It fascinates me quite a bit.  I spent most of the day attempting to
get the following to work with no luck.  I run a linux 2.4.16 kernel, for
future reference.

The jist of my problem is this.  I need to ratelimit an entire interface
(all traffic) to 1.5mbit/sec.  Now, this is obviously really easy and
accurate to do with a tbf qdisc, but when the 1.5mbit gets nailed, things
like ssh tend to suffer a bit with added latency.  Not to mention the game
servers that are run on the box (but that''s not the issue right now,
lets
stick with SSH. ;)

Now, what I want to do is limit the interface to 1.5mbit but ALSO allow
interactive (relying for now on the application to set the correct bits)
precedence over all other traffic.  I.e. ssh traffic.  Obviously a prio rule
here works very well for prioritizing the traffic (or in this case a pfifo
would work just as well), but I can''t limit *all* traffic to 1.5mbit. 
I can
limit each category of traffic to whatever I want, but I can''t have
them
combined.  (i.e. I could "give" ssh 500kbit/sec but I can''t
say "if ssh
isn''t using all that bandwidth, give it up to the other greedy
services").

I looked over cbq a bit, but it seems way too complex, and definitely too
inaccurate to use if I can help it.  But if needed, I''m game.  I also
attempted to get HTB working via the kernel patches, but it seems it
doesn''t
work quite right (if I try to follow the step-by-step examples things start
to error halfway trhough.. it even killed the box once when I hit it with a
lot of data). The reason I want to use the QoS bits is because people scp
large files off this box quite often, and that''s the only way to
distinguish
scp from ssh traffic.

an example is pretty much straight out of the HOWTO:

tc qdisc add dev eth0 root handle 1: prio
tc qdisc add dev eth0 parent 1:1 handle 10: sfq
tc qdisc add dev eth0 parent 1:2 handle 20: tbf rate 1500kbit burst 4000kbit
latency 350ms
tc qdisc add dev eth0 parent 1:3 handle 30: sfq

so there traffic with the bulk transfer bit set gets limited to 1500kbit,
but if there is other traffic it''s left to run free.  I could also
limit the
30 rule, but then if someone is scp''ing a big file, and someone else is
downloading a huge file via http, both limits would be combined and I''d
again be getting hit with a rather large bandwidth bill.

What would be ideal, is a TBF filter on the interface, with a prio filter
behind that.  (prio as opposed to pfifo so I can later add rulesets for
certain TCP ports to give them priority over "everything else"). 
Obviously
you cannot do this however, and this is where I''m getting lost and
hitting
my head against a brick wall.

I guess where I''m getting confused is the whole stateful/unstateful
qdisc
thing.

Anyone have any ideas on how to do this?

(short version: need to ratelimit all traffic to 1.5mbit/sec while giving
interactive QoS bit traffic priority over everything else)

thanks much!  I hope to be around a while here, there is some seriously
interesting stuff.  Makes for some good reading.

-Phil

On Thu, Dec 06, 2001 at 02:58:40PM -0600, Phil Doroff wrote:
> The jist of my problem is this.  I need to ratelimit an entire interface
> (all traffic) to 1.5mbit/sec.  Now, this is obviously really easy and
> accurate to do with a tbf qdisc, but when the 1.5mbit gets nailed, things
> like ssh tend to suffer a bit with added latency.  Not to mention the game
> servers that are run on the box (but that''s not the issue right
now, lets
> stick with SSH. ;)
If you set your limit/latency low enough, latency will be reasonable, but
you can do cooler things:
> Now, what I want to do is limit the interface to 1.5mbit but ALSO allow
> interactive (relying for now on the application to set the correct bits)
> precedence over all other traffic.  I.e. ssh traffic.  Obviously a prio
rule
> here works very well for prioritizing the traffic (or in this case a pfifo
> would work just as well), but I can''t limit *all* traffic to
1.5mbit.  I can
> limit each category of traffic to whatever I want, but I can''t
have them
> combined.  (i.e. I could "give" ssh 500kbit/sec but I
can''t say "if ssh
> isn''t using all that bandwidth, give it up to the other greedy
services").
You could make PRIO qdisc, make a filter that sends up to 25kbits/second of
SSH traffic to band 0, and everything else to band 1. Band 1 would then
contain a TBF set to 1.5mbit/s-25kbit/s.

See
http://ds9a.nl/lartc/HOWTO/cvs/2.4routing/output/2.4routing-12.html#ss12.3
for how to make policing filters. 
> I looked over cbq a bit, but it seems way too complex, and definitely too
> inaccurate to use if I can help it.  But if needed, I''m game.  I
also
> attempted to get HTB working via the kernel patches, but it seems it
doesn''t
> work quite right (if I try to follow the step-by-step examples things start
> to error halfway trhough.. it even killed the box once when I hit it with a
> lot of data). The reason I want to use the QoS bits is because people scp
Devik will be very interested in hearing the details! One crasher bug has
been fixed recently. Please inform him of your problems!
> large files off this box quite often, and that''s the only way to
distinguish
> scp from ssh traffic.
True. You can even match on those with filters, I just added TOS selection
to
  http://ds9a.nl/lartc/HOWTO/cvs/2.4routing/output/2.4routing-9.html#ss9.5
> What would be ideal, is a TBF filter on the interface, with a prio filter
> behind that.  (prio as opposed to pfifo so I can later add rulesets for
> certain TCP ports to give them priority over "everything else"). 
Obviously
You just described HTB :-)
> I guess where I''m getting confused is the whole
stateful/unstateful qdisc
> thing.
Classful/unclassful.
> thanks much!  I hope to be around a while here, there is some seriously
> interesting stuff.  Makes for some good reading.
Just bug Devik about the problems you see with HTB. The more you can help
him, the earlier HTB will be *rock-solid* and in the mainline kernel.

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
''SYN! .. SYN|ACK! .. ACK!'' - the mating call of the internet