LARTC - Jul 2006 - questions about HFSC, VoIP and (dynamic) ingress shaping

Hello,

I just found the great howto and started shaping my internet connection. The 
howto''s last update is a liitle in the past now so I have some
questions
about how things are done the best way nowadays ;-)


To ensure a stable and low latency voip communication I added an HFSC qdisc to 
device ppp0 (1 Mbit SDSL). There are two classes (by now): One for SIP and 
RTP and one for the rest.

Question 1: I defined the voip qdisc as

    tc class add dev ppp0 parent 1:1 classid 1:11 \
        hfsc sc umax 1500b dmax 30ms rate 500kbit ul rate 900kbit

but "tc -s class show dev ppp0" shows

    class hfsc 1:11 parent 1:1 sc m1 0bit d 6.0ms m2 500000bit \
    ul m1 0bit d 0us m2 900000bit

Where does the "0bit d 6.0ms" come from, (what does the other stuff
exactly
mean) and what would be a good setting for voip traffic?




This qdisc only affects outgoing traffic. But I also want to control incoming 
packets and keep the isp''s queue empty.

Question 2: What is the best solution for doing this: ingress qdisc, IMQ, ... 
(thers''s only one link to isp)?



Much traffic on this line is UDP traffic (OpenVPN). 

Question 3: If I do policing on incoming traffic, do UDP transmissions care 
about dropping and reduce transmit speed?




If I begin to control incoming traffic I only want to drop packets that are 
non voip traffic.

Question 4: Is it possible to control what packets are dropped?



I always want to have enough bandwith for incoming voip traffic. But limiting 
the other stuff to static 180 kbit only because voip packets could sometimes 
reach a maximum of 800 kbit sounds not so good.

My idea: A script that periodically (or even better: triggered by asterisk) 
controls the parameters for the ingress shaper (depending on the actual 
upload traffic produced by voip which could easily be measured with tc).

Question 5: Is that possible? Is this necessary at all? Is there already a 
solution?



Regards
Daniel

On Sun, Jul 30, 2006 at 11:09:04PM +0200, Daniel Musketa
wrote:
> Hello,
> 
> I just found the great howto and started shaping my internet connection.
The
> howto''s last update is a liitle in the past now so I have some
questions
> about how things are done the best way nowadays ;-)
> 
> 
> To ensure a stable and low latency voip communication I added an HFSC qdisc
to
> device ppp0 (1 Mbit SDSL). There are two classes (by now): One for SIP and 
> RTP and one for the rest.
> 
> Question 1: I defined the voip qdisc as
> 
>     tc class add dev ppp0 parent 1:1 classid 1:11 \
>         hfsc sc umax 1500b dmax 30ms rate 500kbit ul rate 900kbit
> 
> but "tc -s class show dev ppp0" shows
> 
>     class hfsc 1:11 parent 1:1 sc m1 0bit d 6.0ms m2 500000bit \
>     ul m1 0bit d 0us m2 900000bit
> 
> Where does the "0bit d 6.0ms" come from, (what does the other
stuff exactly
> mean) and what would be a good setting for voip traffic?
> 
> 
> 
> 
> This qdisc only affects outgoing traffic. But I also want to control
incoming
> packets and keep the isp''s queue empty.
> 
> Question 2: What is the best solution for doing this: ingress qdisc, IMQ,
...
> (thers''s only one link to isp)?
> 
If you don''t want to patch the kernel and your machine has only two
network interfaces you can shape the outgoing traffic to the internal interface
instead of the incoming traffic to the internal interface. I have an example
script here:

http://downloads.angulosolido.pt/QoS/HTB_shaper_adv.sh

It uses HTB for shapping though.


> 
> 
> Much traffic on this line is UDP traffic (OpenVPN). 
> 
> Question 3: If I do policing on incoming traffic, do UDP transmissions care
> about dropping and reduce transmit speed?
> 
> 
I don''t think so, unless that is implemented at the application level.
The UDP packets shouldn''t be dropped unless that''s acceptable
to a certain extend (VoIP) or the application resends as necessary.

> 
> 
> If I begin to control incoming traffic I only want to drop packets that are
> non voip traffic.
> 
> Question 4: Is it possible to control what packets are dropped?
> 
Depends if you know the ports they use. If you know it, you just have to mark
them accordingly.

For example with skype it''s a pain in the ass, as it seems not to
respect the port chosen on the configuration. I solved that by reserving
bandwidth per workstation, so that each one get''s TOTAL/N, extensible
to TOTAL of the others don''t request it. For each workstation there is
also a priority rank, but for cases like skype, whose ports we can''t
control, it ends up being up to each user to ensure there''s enough
bandwidth on her share for skype to work.

Take a look at the link I pasted above.
> 
> 
> I always want to have enough bandwith for incoming voip traffic. But
limiting
> the other stuff to static 180 kbit only because voip packets could
sometimes
> reach a maximum of 800 kbit sounds not so good.
> 
Sure!
> My idea: A script that periodically (or even better: triggered by asterisk)
> controls the parameters for the ingress shaper (depending on the actual 
> upload traffic produced by voip which could easily be measured with tc).
> 
> Question 5: Is that possible? Is this necessary at all? Is there already a 
> solution?
> 
I don''t think you need this. A setup with HTB solves this problem,
since each traffic class has a defined RATE as well as a defined CEIL rate,
which it will take whenever available.

I can''t comment about HFSC as I never used it. 

Best regards
Gustavo
> 
> 
> Regards
> Daniel
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
-- 
Angulo Sólido - Tecnologias de Informação
http://angulosolido.pt

On Sun, Jul 30, 2006 at 11:52:48PM +0100, Gustavo Homem
wrote:
> On Sun, Jul 30, 2006 at 11:09:04PM +0200, Daniel Musketa wrote:
> > Hello,
> > 
> > I just found the great howto and started shaping my internet
connection. The
> > howto''s last update is a liitle in the past now so I have
some questions
> > about how things are done the best way nowadays ;-)
> > 
> > 
> > To ensure a stable and low latency voip communication I added an HFSC
qdisc to
> > device ppp0 (1 Mbit SDSL). There are two classes (by now): One for SIP
and
> > RTP and one for the rest.
> > 
> > Question 1: I defined the voip qdisc as
> > 
> >     tc class add dev ppp0 parent 1:1 classid 1:11 \
> >         hfsc sc umax 1500b dmax 30ms rate 500kbit ul rate 900kbit
> > 
> > but "tc -s class show dev ppp0" shows
> > 
> >     class hfsc 1:11 parent 1:1 sc m1 0bit d 6.0ms m2 500000bit \
> >     ul m1 0bit d 0us m2 900000bit
> > 
> > Where does the "0bit d 6.0ms" come from, (what does the
other stuff exactly
> > mean) and what would be a good setting for voip traffic?
> > 
> > 
> > 
> > 
> > This qdisc only affects outgoing traffic. But I also want to control
incoming
> > packets and keep the isp''s queue empty.
> > 
> > Question 2: What is the best solution for doing this: ingress qdisc,
IMQ, ...
> > (thers''s only one link to isp)?
> > 
> 
> If you don''t want to patch the kernel and your machine has only
two network interfaces you can shape the outgoing traffic to the internal
interface instead of the incoming traffic to the internal interface. I have an
example script here:
"you can shape the outgoing traffic to the internal interface instead of
the incoming traffic to the *external* interface"
> 
> http://downloads.angulosolido.pt/QoS/HTB_shaper_adv.sh
> 
> It uses HTB for shapping though.
> 
> 
> 
> > 
> > 
> > Much traffic on this line is UDP traffic (OpenVPN). 
> > 
> > Question 3: If I do policing on incoming traffic, do UDP transmissions
care
> > about dropping and reduce transmit speed?
> > 
> > 
> 
> I don''t think so, unless that is implemented at the application
level. The UDP packets shouldn''t be dropped unless that''s
acceptable to a certain extend (VoIP) or the application resends as necessary.
> 
> 
> > 
> > 
> > If I begin to control incoming traffic I only want to drop packets
that are
> > non voip traffic.
> > 
> > Question 4: Is it possible to control what packets are dropped?
> > 
> 
> Depends if you know the ports they use. If you know it, you just have to
mark them accordingly.
> 
> For example with skype it''s a pain in the ass, as it seems not to
respect the port chosen on the configuration. I solved that by reserving
bandwidth per workstation, so that each one get''s TOTAL/N, extensible
to TOTAL of the others don''t request it. For each workstation there is
also a priority rank, but for cases like skype, whose ports we can''t
control, it ends up being up to each user to ensure there''s enough
bandwidth on her share for skype to work.
> 
> Take a look at the link I pasted above.
> 
> > 
> > 
> > I always want to have enough bandwith for incoming voip traffic. But
limiting
> > the other stuff to static 180 kbit only because voip packets could
sometimes
> > reach a maximum of 800 kbit sounds not so good.
> > 
> 
> Sure!
> 
> > My idea: A script that periodically (or even better: triggered by
asterisk)
> > controls the parameters for the ingress shaper (depending on the
actual
> > upload traffic produced by voip which could easily be measured with
tc).
> > 
> > Question 5: Is that possible? Is this necessary at all? Is there
already a
> > solution?
> > 
> 
> I don''t think you need this. A setup with HTB solves this problem,
since each traffic class has a defined RATE as well as a defined CEIL rate,
which it will take whenever available.
> 
> I can''t comment about HFSC as I never used it. 
> 
> Best regards
> Gustavo
> 
> > 
> > 
> > Regards
> > Daniel
> > _______________________________________________
> > LARTC mailing list
> > LARTC@mailman.ds9a.nl
> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
> 
> -- 
> Angulo Sólido - Tecnologias de Informação
> http://angulosolido.pt
> 
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
-- 
Angulo Sólido - Tecnologias de Informação
http://angulosolido.pt

Hello, Gustavo,

thank you for your answer.

Am Montag, 31. Juli 2006 00:52 schrieb Gustavo Homem:
> On Sun, Jul 30, 2006 at 11:09:04PM +0200, Daniel Musketa wrote:
[...]
> > This qdisc only affects outgoing traffic. But I also want to control
> > incoming packets and keep the isp''s queue empty.
> >
> > Question 2: What is the best solution for doing this: ingress qdisc,
IMQ,
> > ... (thers''s only one link to isp)?
>
> If you don''t want to patch the kernel and your machine has only
two network
> interfaces you can shape the outgoing traffic to the internal interface
> instead of the incoming traffic to the internal interface. I have an
> example script here:
>
> http://downloads.angulosolido.pt/QoS/HTB_shaper_adv.sh
>
> It uses HTB for shapping though.

Sorry, I should have explained a little more detailed: (Nearly) all download 
traffic is initiated by the machine itself, so I can''t use the
"limiting the
clients" trick here.



> > Much traffic on this line is UDP traffic (OpenVPN).
> >
> > Question 3: If I do policing on incoming traffic, do UDP transmissions
> > care about dropping and reduce transmit speed?
>
> I don''t think so, unless that is implemented at the application
level. The
> UDP packets shouldn''t be dropped unless that''s acceptable
to a certain
> extend (VoIP) or the application resends as necessary.
I am sure OpenVPN resends dropped packets (sooner or later) but don''t
know yet
if it adapts its transmit rate to the rate of the reply packets ...




> > If I begin to control incoming traffic I only want to drop packets
that
> > are non voip traffic.
> >
> > Question 4: Is it possible to control what packets are dropped?
>
> Depends if you know the ports they use. If you know it, you just have to
> mark them accordingly.
>
> For example with skype it''s a pain in the ass, 
The only voip traffic (over ppp0) is produced by an asterisk server (on the 
same machine). I know all used ports (SIP and RTP) and for outgoing packets I 
already mark them with iptables.



> > My idea: A script that periodically (or even better: triggered by
> > asterisk) controls the parameters for the ingress shaper (depending on
> > the actual upload traffic produced by voip which could easily be
measured
> > with tc).
> >
> > Question 5: Is that possible? Is this necessary at all? Is there
already
> > a solution?
>
> I don''t think you need this. A setup with HTB solves this problem,
since
> each traffic class has a defined RATE as well as a defined CEIL rate, which
> it will take whenever available.
But then I need IMQ and a patched kernel, right?

> Best regards
> Gustavo
Muito obrigado pela sua ajuda ...
Daniel

Hello Daniel,
> >
> > If you don''t want to patch the kernel and your machine has
only two
> > network interfaces you can shape the outgoing traffic to the internal
> > interface instead of the incoming traffic to the internal interface. I
> > have an example script here:
> >
> > http://downloads.angulosolido.pt/QoS/HTB_shaper_adv.sh
> >
> > It uses HTB for shapping though.
>
> Sorry, I should have explained a little more detailed: (Nearly) all
> download traffic is initiated by the machine itself, so I can''t
use the
> "limiting the clients" trick here.
Ah... I see...
> > > Question 4: Is it possible to control what packets are dropped?
> >
> > Depends if you know the ports they use. If you know it, you just have
to
> > mark them accordingly.
> >
> > For example with skype it''s a pain in the ass,
>
> The only voip traffic (over ppp0) is produced by an asterisk server (on the
> same machine). I know all used ports (SIP and RTP) and for outgoing packets
> I already mark them with iptables.
>
So you can match the to the appropriate class.
> > > My idea: A script that periodically (or even better: triggered by
> > > asterisk) controls the parameters for the ingress shaper
(depending on
> > > the actual upload traffic produced by voip which could easily be
> > > measured with tc).
> > >
> > > Question 5: Is that possible? Is this necessary at all? Is there
> > > already a solution?
> >
> > I don''t think you need this. A setup with HTB solves this
problem, since
> > each traffic class has a defined RATE as well as a defined CEIL rate,
> > which it will take whenever available.
>
> But then I need IMQ and a patched kernel, right?
For the script I pointed you don''t need it, because HTB is part of the 
standard kernel.

However for your case, where most of the incoming traffic goes directly to the 
router machine I think so.

If you decide to go that way, take a look at this howto:

http://www.tldp.org/HOWTO/ADSL-Bandwidth-Management-HOWTO/index.html

Boa sorte!
Gustavo
>
> > Best regards
> > Gustavo
>
> Muito obrigado pela sua ajuda ...
> Daniel
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
-- 
 Angulo Sólido - Tecnologias de Informação
http://angulosolido.pt

Daniel Musketa wrote:
> Hello,
> 
> I just found the great howto and started shaping my internet connection.
The
> howto''s last update is a liitle in the past now so I have some
questions
> about how things are done the best way nowadays ;-)
> 
> 
> To ensure a stable and low latency voip communication I added an HFSC qdisc
to
> device ppp0 (1 Mbit SDSL). There are two classes (by now): One for SIP and 
> RTP and one for the rest.
> 
> Question 1: I defined the voip qdisc as
> 
>     tc class add dev ppp0 parent 1:1 classid 1:11 \
>         hfsc sc umax 1500b dmax 30ms rate 500kbit ul rate 900kbit
> 
> but "tc -s class show dev ppp0" shows
> 
>     class hfsc 1:11 parent 1:1 sc m1 0bit d 6.0ms m2 500000bit \
>     ul m1 0bit d 0us m2 900000bit
> 
> Where does the "0bit d 6.0ms" come from, (what does the other
stuff exactly
> mean) and what would be a good setting for voip traffic?
I still get confused thinking about hfsc and the examples I''ve seen. I 
think it''s because I can''t get away from thinking about what
really
happens and the numbers like dmax must relate to something other than 
max delay, which will be related to your setup - hmm I suppose you 
should change the numbers to fit.... - see I''m confused again :-) Also 
the original paper and bsd docs relate to something different to linux 
hfsc which has link share and upper limit curves (which make it alot 
more useful). Just take it that any suggestions I make are probably 
wrong :-)

You can specify the settings for a class in two ways and tc shows only 
one of them.

dmax 30ms @ 500 kbit with 1500 packet - the bitrate latency of a 1500 
byte packet @ 500kbit is 24ms you say 30 so there is 6ms "spare" so a 
plot of that curve will 6ms at 0bit and then 500kbit. This is more like 
I would have for a bulk class - which I would also make ls not rt/sc.

Since this is voip I don''t think you''ll need anything like
1500 byte anyway.

For rt/sc you can - as long as you adjust the other curves you can have 
an m1 of  say 1mbit (ie link speed or if you have more than 1 rt link 
speed/number of rt classes) and an m2 of the long term rate like 500kbit 
- this I think is how a class for rt traffic should be specified.

> This qdisc only affects outgoing traffic. But I also want to control
incoming
> packets and keep the isp''s queue empty.
> 
> Question 2: What is the best solution for doing this: ingress qdisc, IMQ,
...
> (thers''s only one link to isp)?
Ingress shaping (by which I mean where you are the wrong end of a 
bottleneck) is not totally doable like egress - depends on link speed, 
requirements and what the users do to some extent. You will need to 
sacrifice bandwidth whether you police or shape - say 20% to 50%. You 
will need to keep buffers (virtual or real fairly short - headdropping 
would be nicer than tail for real queues, but you can''t without hacking
code) and waste bandwidth by dropping packets that already used your 
link (though you can think of these as coming out of what you already 
sacrificed, you will still be billed/metered if you pay like that). 
There are other tweaks you can try eg when I had a 500kbit link, getting 
out of tcp slowstart using connbytes to mark and sending first XkB bulk 
to a shorter lower rate queue than normal.

IFB is an in kernel replacement for IMQ - there are still things you 
can''t do with it or policers eg if you are doing nat and need to get
the
ingress traffic after it''s been denatted by netfilter.
> 
> Much traffic on this line is UDP traffic (OpenVPN). 
> 
> Question 3: If I do policing on incoming traffic, do UDP transmissions care
> about dropping and reduce transmit speed?
> 
Yes if it''s tcp in udp for vpn then it will behave like tcp - for other
udp it will be app specific.
> 
> If I begin to control incoming traffic I only want to drop packets that are
> non voip traffic.
> 
> Question 4: Is it possible to control what packets are dropped?
> 
Yes - well maybe, if you use policers + actions you can choose what 
happens to overrate traffic. I think you will need to test what really 
happens, though - maybe it will work out on average rather than for 
definate (I am thinking that the overlimits traffic does not get 
accounted for by meters/ rate estimators).
> 
> 
> I always want to have enough bandwith for incoming voip traffic. But
limiting
> the other stuff to static 180 kbit only because voip packets could
sometimes
> reach a maximum of 800 kbit sounds not so good.
> 
> My idea: A script that periodically (or even better: triggered by asterisk)
> controls the parameters for the ingress shaper (depending on the actual 
> upload traffic produced by voip which could easily be measured with tc).
> 
> Question 5: Is that possible? Is this necessary at all? Is there already a 
> solution?
Could be one way to do it, though it maybe unneccesary depending on what 
you do eg ifb or policers - in fact I think both could work without 
scripting. One bonus of scripting would be if you can change things 
before the new ingress traffic starts - queueing/policing only react 
when it''s too late so it''s really hard to avoid brief (maybe 
insignificant for most people) latency bursts (this applies to every new 
tcp connection aswell).

Andy.