LARTC - Jun 2001 - PRIOMAP, SPLIT, DEFMAP usage info + PRIO, TBF and other bugs

this message was originally send to stafke@iname.com
http://users.belgacom.net/staf/

Currently I''m trying to install all this QoS stuff on my bridge and I
have A LOT of problems,
so I very glad to find someone interested in helping others to solve this not
trivial problem.

I decided to share that I have found so far with you :)
NOTE: DON''T consider all this correct, it''s all just my
thoughts. So don''t hit me if I''m wrong.

From you TODO list:

1. Unclassified traffic - traffic to which none of the filters has been applyed,
so by default it
    will fall to root class. You can use Defmap to assign all this unclassified
traffic to other
    class. For example setting Defmap to 0xFFFF will assign all unclassified by
other means
    traffic to this class. Split also must be specified else defmap will not
work.
    Here is some words from Alex:

"defmaps are intended to make fallback classification
when a packet does not match to any stronger classifier
or no classifiers are configured.

Algorithm is very simple: protocol layer assigns a logical priority
numbered of 0 to 15 to any packet. For forwarded packets it is derived
from tos, for locally originated ones it is controlled by SO_PRIORITY
socket option (or derived from tos too). 

If a class is ready to serve some logical priority,
it has corresponding bit in its defmap.

If packet matches to any classifier, this priority
is ignored. When it does not, cbq searches for class
which is ready to serve corresponding priority level.
That''s almost all. Note that, to make cbq working properly,
you should sweep at least all priorities 0..7, otherwise
unclassified packets will fallback to root class,
which is not intended to transmit anything at all.

For IPv4 mapping tos -> priority is established in route.c
by array ip_tos2prio[]. It is not changable from user level for now
and complies to canonical ietf rules. You may change it to
diffserv, if you want.

About "split". Normally, split is root.

You may set split to a node in hierarchy and use some simple
and fast classifier, which understands only limited set of keys
(sort of route), to point to this node.
In this case, only classes with split==this node,
will be matched. It means, that sk->priority and tos
may live in peace with multiagency link sharing
with zero classification cost. Seems, it is
the best my invention for last year 8)"

Ok. So far so good. I still not sleeping :)

2. About this Priomap. This all about this early mentioned "logical
priority",
    if we have for example Priomap 0 0 0 0 0  1 1 1 1 1 2 2 2 2 2 2
    then all traffic with any of the 5 high bits set in this "logical
priority" will be
    assigned to band 0, if any of the 5 bits set in the middle then this packet
    will be assigned to band 1 and so forth.

Oh, one more notice:
    AS FAR I UNDERSTAND PRIO NOT WORKING AT ALL, OR AT LEAST
    NOT WORKING AS IT MUST WORK.
    First of all it not implement class manipulation functions, so if you just
try
    to list PRIO classes (tc class ls dev eth0) this will give you segmentation
    fault :(. The more the better. I far as I understand PRIO qdisc creates
classes
    "on fly", so I tryed to filter some packets to this classed. After
2 weaks of
    experements I understand that it don''t prioritize packets AT ALL.
(I''ll be glad
    to hear that I''m wrong...)

    OK some more bugs....

    TBF. If you defined this one as described in README it will not work :(
    At least it''s not working on my setup.

    tc qdisc add dev eth1 parent 1:2 tbf rate 64Kbit buffer 5Kb/8 limit 10Kb

    You''ll get latency set to something near 5000s :(
    So, to work correctly you must define NOT LIMIT but LATENCY (1000 = 1.2 ms),
    or define BURST and use it with LIMIT, BTW you can''t use Kb in
burst you must
    use kB :(

    I think this is enough. I''m very interested in you latency testing.
How you manage
    to do some latency changing with CBQ. AFAIK CBQ not intended to PRIORITIZE 
    traffic it only Fair Destribute it in case of congestion.

    Hope for you understanding. And excuse my english. I''m russian too
;)
    
    Plz share you experience with the comunity!
    Alex where is fixed version? And where is HPFQ support? Where? Aaaa? Gde?
    --
    Ness
    ness@inbox.lv

> ness wrote:
> Oh, one more notice:
>     AS FAR I UNDERSTAND PRIO NOT WORKING AT ALL, OR AT LEAST
>     NOT WORKING AS IT MUST WORK.
>     First of all it not implement class manipulation functions, so if
> you just try
>     to list PRIO classes (tc class ls dev eth0) this will give you
> segmentation
>     fault :(. The more the better. I far as I understand PRIO qdisc
> creates classes
>     "on fly", so I tryed to filter some packets to this classed.
After
> 2 weaks of
the advanced networking howto
(http://qos.ittc.ukans.edu/howto/node17.html) says PRIO does not support
clasess:

"Not all queuing disciplines support classes. The ones that support
classes include the CBQ, the DS_MARK, the CSZ and
the p-FIFO queuing disciplines. The rest of the queuing disciplines do
not support classes. "

but the code net/sched/sch_prio.c has prio_classify() call .... I think
the howto is too old, PRIO supports classes .....
>     experements I understand that it don''t prioritize packets AT
ALL.
> (I''ll be glad
>     to hear that I''m wrong...)
> 
does not in my tests, I am also looking why ....


Nikolai

On Mon, 02 Jul 2001 17:09:22 -0500
Nikolai Vladychevski <niko@netspace.com.mx> wrote:
> > ness wrote:
> >     experements I understand that it don''t prioritize packets
AT ALL.
> > (I''ll be glad
> >     to hear that I''m wrong...)
> > 
> 
> does not in my tests, I am also looking why ....
i was playing with net/sched/sch_prio.c, trying to analize what it does
and modifying it just for write some crap in syslog and debug

i''ve done a few tests, ie doing a heavy upload and ping another machine
over internet, putting icmp packets into band 0 of the prio qdisc, and the
rest into band 1, and what i see is that the prio qdisc actually
classifies packets into its bands according to filters you define, and in
fact dequeues first all packets from the bands with the most priority...
but the final result is not as expected, and i have the same delay as if i
don''t use any qos... i''ve tried attaching some different qdisc
to the
bands of the prio qdisc: tbf, sfq, just fifos...

it seems to me its that prio does its work just ok, but the problem is
that the networking code is making dequeue the packets more fast than the
link can transmit them, without care about if they are accumulating again
at some point after

-- 

Ivan Lopez de Pedro
aSkAiNeT
Proud member of The Gey Panda Crew
Running Debian Sid GNU/Linux

Ivan Lopez wrote:
> 
> On Mon, 02 Jul 2001 17:09:22 -0500
> Nikolai Vladychevski <niko@netspace.com.mx> wrote:
> 
> > > ness wrote:
> > >     experements I understand that it don''t prioritize
packets AT ALL.
> > > (I''ll be glad
> > >     to hear that I''m wrong...)
> > >
> >
> > does not in my tests, I am also looking why ....
> 
> i was playing with net/sched/sch_prio.c, trying to analize what it does
> and modifying it just for write some crap in syslog and debug
> 
> i''ve done a few tests, ie doing a heavy upload and ping another
machine
> over internet, putting icmp packets into band 0 of the prio qdisc, and the
> rest into band 1, and what i see is that the prio qdisc actually
> classifies packets into its bands according to filters you define, and in
> fact dequeues first all packets from the bands with the most priority...
> but the final result is not as expected, and i have the same delay as if i
> don''t use any qos... i''ve tried attaching some different
qdisc to the
> bands of the prio qdisc: tbf, sfq, just fifos...
> 
> it seems to me its that prio does its work just ok, but the problem is
> that the networking code is making dequeue the packets more fast than the
> link can transmit them, without care about if they are accumulating again
> at some point after
> 
I just found a nice paper with explanations about PRIO qdisc with
examples an figures, its in Chapter 7:

http://dcn.ssu.ac.kr/~softgear/prog_sw_report_summer_99.pdf



Nikolai

Ivan Lopez wrote:
> 
> On Mon, 02 Jul 2001 17:09:22 -0500
> Nikolai Vladychevski <niko@netspace.com.mx> wrote:
> 
> > > ness wrote:
> > >     experements I understand that it don''t prioritize
packets AT ALL.
> > > (I''ll be glad
> > >     to hear that I''m wrong...)
> > >
> >
> > does not in my tests, I am also looking why ....
> 
> i was playing with net/sched/sch_prio.c, trying to analize what it does
> and modifying it just for write some crap in syslog and debug
> 
> i''ve done a few tests, ie doing a heavy upload and ping another
machine
> over internet, putting icmp packets into band 0 of the prio qdisc, and the
> rest into band 1, and what i see is that the prio qdisc actually
> classifies packets into its bands according to filters you define, and in
> fact dequeues first all packets from the bands with the most priority...
> but the final result is not as expected, and i have the same delay as if i
> don''t use any qos... i''ve tried attaching some different
qdisc to the
> bands of the prio qdisc: tbf, sfq, just fifos...
> 
> it seems to me its that prio does its work just ok, but the problem is
> that the networking code is making dequeue the packets more fast than the
> link can transmit them, without care about if they are accumulating again
> at some point after
> 
This is true. PRIO works. The following is my setup for a LAN that is
masqueraded on a 56K modem link:

#! /bin/bash -x

tc qdisc add dev eth0 root handle 1: prio
tc qdisc add dev eth0 parent 1:1 handle 2: tbf limit 1600 burst 2600
rate 28800
tc filter add dev eth0 parent 1:0 protocol ip prio 1 u32 match ip dst
192.168.1.2 classid 1:1
tc filter add dev eth0 parent 1:0 protocol ip prio 7 u32 match ip dst
192.168.1.4 classid 1:1

iptables -A PREROUTING -i eth0 -s 192.168.1.2 -t mangle -j MARK
--set-mark 2
iptables -A PREROUTING -i eth0 -s 192.168.1.4 -t mangle -j MARK
--set-mark 4

tc qdisc add dev ppp0 root handle 2: prio
tc qdisc add dev ppp0 parent 2:1 handle 3: tbf limit 1600 burst 2600
rate 28800
tc filter add dev ppp0 protocol ip parent 2:0 prio 1 handle 2 fw classid
2:1
tc filter add dev ppp0 protocol ip parent 2:0 prio 7 handle 4 fw classid
2:1


This script creates a virutal channel on eth0 that somewhat equal to 56K
modem (well, I have to tune the TBF , did not yet finished that part)  ,
so when prio dequeues the packets TBF just drops those that are
overloading the bandwidth..... 

The ip 192.168.1.4 has the highest priority (band 0) , the 192.168.1.2
the lowest (band 2)
I tested this with FTP and Web. Since it''s a strict priority scheduler,
if both machines make the download over ftp, the downloads for
192.168.1.2 just stop after some time and 192.168.1.4 keeps downloading
at full. When you stop downloading on 192.168.1.4 and the connection
isn''t timed out by ftp client on the machine .2, it returns to the
normal download rate. 

The only thing left to do is tune the TBF correctly to be equal to the
bandwidth of the outgoing link ...

Regards
Nikolai