LARTC - Apr 2004 - newbie: TC[NG] with (256kbit/s down and 768kbit/s up) on a router

Hi all,

this is really not really very easy to understand, or, to get in.

Well, I''ve the following configuration on the router box:

LAN
  - interface: eth0
  - network: 192.168.2.5/24
  - bandwidth: 100Mbit/s
INET interface
  - interface: ppp0
  - network: .dynamic.ip./0
  - bandwidth: DOWN=1536kbit/s and UP=256kbit/s

the LAN interface is to serve 6 other clients with internet and local
services. My goal NOW was, or is, to garrantie each client with a fair
amount of bandwith for both, up and down.

That is, each client inside the LAN should get garrantied
  - PER_CLIENT_DOWN=256kbit/s
  - and PER_CLIENT_UP=42kbit/s
Each unused bandwith may be shared between them.

The LAN clients have IP pool:
    192.168.2.2-192.168.2.4, and
    192.168.2.6-192.168.2.8

But how exactly do I now express my wish in TCNG syntax?

Some kind of pseudo code like below...

device ppp0 {
    input ( 1536kbit/s ) { // upstream
        class ( 256kbit/s; may borrow ) { catch ip 192.168.2.2; }
        class ( 256kbit/s; may borrow ) { catch ip 192.168.2.3; }
        class ( 256kbit/s; may borrow ) { catch ip 192.168.2.4; }
        class ( 256kbit/s; may borrow ) { catch ip 192.168.2.6; }
        class ( 256kbit/s; may borrow ) { catch ip 192.168.2.7; }
        class ( 256kbit/s; may borrow ) { catch ip 192.168.2.8; }
    }
    output ( 256kbit/s ) { // downstream
        class ( 42kbit/s; may borrow ) { catch ip 192.168.2.2; }
        class ( 42kbit/s; may borrow ) { catch ip 192.168.2.3; }
        class ( 42kbit/s; may borrow ) { catch ip 192.168.2.4; }
        class ( 42kbit/s; may borrow ) { catch ip 192.168.2.6; }
        class ( 42kbit/s; may borrow ) { catch ip 192.168.2.7; }
        class ( 42kbit/s; may borrow ) { catch ip 192.168.2.8; }
    }
}

The "device" object is meant to represent the device''s
configuration
specific data. "input" as child of "device" represents the
input
bandwidth configuration - same goes for "output". class is just like
tc/tcng, I guess. "catch ip IP" just tells, what IP packets should be
queued in this class. The queuing discipline to be used is rarely
unimportant, maybe htb, cbq, or tbf, what ever(?) best fits right here.

Sorry, this is *my* brain-dead-pseudo-code to explain, what I want, with a
syntax associated to the tcc(tcng) examples I have found on the net.

Could someone *now* show me, how my goal should look in tcng syntax?

Many thanks,
Christian Parpart.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

On Friday 23 April 2004 21:23, trapni@surakware.net
wrote:
> Hi all,
Hello.
> this is really not really very easy to understand, or, to get in.
I spent several weeks playing to tcng.  I found a few useful references.

http://tldp.org/HOWTO/Traffic-Control-tcng-HTB-HOWTO/index.html
http://mailman.ds9a.nl/pipermail/lartc/2003q4/010826.html
http://linux-ip.net/gl/tcng/tcng.html
> Well, I''ve the following configuration on the router box:
>
> LAN
>   - interface: eth0
>   - network: 192.168.2.5/24
>   - bandwidth: 100Mbit/s
> INET interface
>   - interface: ppp0
>   - network: .dynamic.ip./0
>   - bandwidth: DOWN=1536kbit/s and UP=256kbit/s
>
> the LAN interface is to serve 6 other clients with internet and local
> services. My goal NOW was, or is, to garrantie each client with a fair
> amount of bandwith for both, up and down.
Egress is easy.  Ingress seems to be a topic that is discussed often on LARTC, 
and I believe your options are to either use an ingress policer or the IMQ 
target.  The former you can do directly with tcng, the latter I believe you 
cannot.
> That is, each client inside the LAN should get garrantied
>   - PER_CLIENT_DOWN=256kbit/s
>   - and PER_CLIENT_UP=42kbit/s
> Each unused bandwith may be shared between them.
>
> The LAN clients have IP pool:
>     192.168.2.2-192.168.2.4, and
>     192.168.2.6-192.168.2.8
>
> But how exactly do I now express my wish in TCNG syntax?
>
> Some kind of pseudo code like below...
>
<snip>
> The "device" object is meant to represent the device''s
configuration
> specific data. "input" as child of "device" represents
the input
> bandwidth configuration - same goes for "output". class is just
like
> tc/tcng, I guess. "catch ip IP" just tells, what IP packets
should be
> queued in this class. The queuing discipline to be used is rarely
> unimportant, maybe htb, cbq, or tbf, what ever(?) best fits right here.
You''d probably use HTB for egress.  If you decide to use IMQ you might
use it
in both directions.
> Sorry, this is *my* brain-dead-pseudo-code to explain, what I want, with a
> syntax associated to the tcc(tcng) examples I have found on the net.
>
> Could someone *now* show me, how my goal should look in tcng syntax?
I don''t think you can use IMQ from within tcng, so you may not be able
to do
ingress and egress with a single tool.
> Many thanks,
> Christian Parpart.
-- 

Jason Boxman
Perl Programmer / *NIX Systems Administrator
Shimberg Center for Affordable Housing | University of Florida
http://edseek.com/ - Linux and FOSS stuff

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Jason Boxman wrote:
> Egress is easy.  Ingress seems to be a topic that is discussed often on
LARTC,
> and I believe your options are to either use an ingress policer or the IMQ 
> target.  The former you can do directly with tcng, the latter I believe you
> cannot.
I know nothing about TCNG so can''t help there.

You can shape ingress without using IMQ as long as you have just one LAN 
interface and don''t care about traffic headed for the shaping PC. You 
just shape on the LAN interface.

Andy.


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sunday 25 April 2004 09:06, Andy Furniss wrote:
> Jason Boxman wrote:
> > Egress is easy.  Ingress seems to be a topic that is discussed often
on
> > LARTC, and I believe your options are to either use an ingress policer
or
> > the IMQ target.  The former you can do directly with tcng, the latter
I
> > believe you cannot.
>
> I know nothing about TCNG so can''t help there.
>
> You can shape ingress without using IMQ as long as you have just one LAN
> interface and don''t care about traffic headed for the shaping PC.
You
> just shape on the LAN interface.
But *how* does such a setup now looks like, either in tcng or in gc syntax?

This is what I actually do:
- --------------------------------------------------
#! /bin/sh

DEV=ppp0
UP=256
DOWN=768
CLIENTS="192.168.2.1 192.168.2.2 192.168.2.3 192.168.2.5 192.168.2.6
192.168.2.7 192.168.2.8"
TC=$(which tc)

# reset
$TC qdisc del dev ${DEV} root &>/dev/null
$TC qdisc del dev ${DEV} ingress &>/dev/null

# attach HTB queue discipline to device $DEV
$TC qdisc add dev $DEV root handle 1: htb default 12

# create client classes for shaping DOWN-stream
crate=$[DOWN / NumClients]
i=0
for host in $CLIENTS; do
  $TC class add dev $DEV parent 1:1 classid 1:1$i htb rate ${crate}kbit ceil
${DOWN}kbit
  $TC filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src $host
flowid 1:1$i
  i=$[i + 1]
done

# TODO shaping UP stream
- --------------------------------------------------

This is my script. And I do not really now, *where* to differ
here to once shape down-stream, and once to shape the up-stream

I''d be really really very happy, if someone would point
me in this *wrong* script to the right direction.

Many thanks,
Christian Parpart.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFAi/jRPpa2GmDVhK0RAiyiAJ9t1LngvstQqwqGkTC367USYfcQtQCeNHUV
nc9176QOuUWp1XqeCSrbj8g=Po1b
-----END PGP SIGNATURE-----
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Christian Parpart wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Sunday 25 April 2004 09:06, Andy Furniss wrote:
> 
>>Jason Boxman wrote:
>>
>>>Egress is easy.  Ingress seems to be a topic that is discussed often
on
>>>LARTC, and I believe your options are to either use an ingress
policer or
>>>the IMQ target.  The former you can do directly with tcng, the
latter I
>>>believe you cannot.
>>
>>I know nothing about TCNG so can''t help there.
>>
>>You can shape ingress without using IMQ as long as you have just one LAN
>>interface and don''t care about traffic headed for the shaping
PC. You
>>just shape on the LAN interface.
> 
> 
> But *how* does such a setup now looks like, either in tcng or in gc syntax?
> 
> This is what I actually do:
> - --------------------------------------------------
> #! /bin/sh
> 
> DEV=ppp0
> UP=256
> DOWN=768
> CLIENTS="192.168.2.1 192.168.2.2 192.168.2.3 192.168.2.5 192.168.2.6
192.168.2.7 192.168.2.8"
> TC=$(which tc)
> 
> # reset
> $TC qdisc del dev ${DEV} root &>/dev/null
> $TC qdisc del dev ${DEV} ingress &>/dev/null
> 
> # attach HTB queue discipline to device $DEV
> $TC qdisc add dev $DEV root handle 1: htb default 12
> 
> # create client classes for shaping DOWN-stream
> crate=$[DOWN / NumClients]
> i=0
> for host in $CLIENTS; do
>   $TC class add dev $DEV parent 1:1 classid 1:1$i htb rate ${crate}kbit
ceil ${DOWN}kbit
>   $TC filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src
$host flowid 1:1$i
>   i=$[i + 1]
> done
> 
> # TODO shaping UP stream
> - --------------------------------------------------
> 
> This is my script. And I do not really now, *where* to differ
> here to once shape down-stream, and once to shape the up-stream
> 
> I''d be really really very happy, if someone would point
> me in this *wrong* script to the right direction.
> 
You have to set you rates lower than your real rates - for ingress about 
80% so you actually get queues growing that you can control. For egress 
about 85% with dsl as there are extra overheads and TC counts IP size.

You should be shaping on eth0 if that''s your LAN facing interface - you
shape egress from the shaping box to the LAN to do ingress (on simple 
setups). The src IP match needs to change to dst.

As it is the script may have too big queues - but should work as a test, 
you may also endup wanting to split interactive traffic from bulk to 
make things nicer for users - but that sort of thing is policy to be 
thought about/agreed by users.

Andy.


> Many thanks,
> Christian Parpart.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
> 
> iD8DBQFAi/jRPpa2GmDVhK0RAiyiAJ9t1LngvstQqwqGkTC367USYfcQtQCeNHUV
> nc9176QOuUWp1XqeCSrbj8g> =Po1b
> -----END PGP SIGNATURE-----
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> 

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 26 April 2004 10:01, Andy Furniss wrote:
> > On Sunday 25 April 2004 09:06, Andy Furniss wrote:
> >>Jason Boxman wrote:
> >>>Egress is easy.  Ingress seems to be a topic that is discussed
often on
> >>>LARTC, and I believe your options are to either use an ingress
policer
> >>> or the IMQ target.  The former you can do directly with tcng,
the
> >>> latter I believe you cannot.
> >>
> >>I know nothing about TCNG so can''t help there.
> >>
> >>You can shape ingress without using IMQ as long as you have just
one LAN
> >>interface and don''t care about traffic headed for the
shaping PC. You
> >>just shape on the LAN interface.
> >
> > But *how* does such a setup now looks like, either in tcng or in gc
> > syntax?
> >
> > This is what I actually do:
[...zap...]
> >
> > This is my script. And I do not really now, *where* to differ
> > here to once shape down-stream, and once to shape the up-stream
> >
> > I''d be really really very happy, if someone would point
> > me in this *wrong* script to the right direction.
>
> You have to set you rates lower than your real rates - for ingress about
> 80% so you actually get queues growing that you can control. For egress
> about 85% with dsl as there are extra overheads and TC counts IP size.
thx.
> You should be shaping on eth0 if that''s your LAN facing interface
- you
> shape egress from the shaping box to the LAN to do ingress (on simple
> setups). The src IP match needs to change to dst.
>
> As it is the script may have too big queues - but should work as a test,
> you may also endup wanting to split interactive traffic from bulk to
> make things nicer for users - but that sort of thing is policy to be
> thought about/agreed by users.
This is all nice, but, I''d be happy to see some *working* example code.
That''s
why I posted my *wrong* setup, possible to point me to the right direction, 
by showing me, *what* I did wrong.

Could someone show me some simple example code for incress+egress shaping for 
ppp0 (for a router with clients at eth0)?

thanks in advance,
Christian Parpart.
> Andy.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFAj25nPpa2GmDVhK0RAg4DAJ9AQAGZgbD1UhP95azObPzsi8kvaQCeLvsC
q2ELEmtQPTKWuVZu1GM7VfU=iIPw
-----END PGP SIGNATURE-----
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Am Wednesday 28 April 2004 10:42 schrieb Christian
Parpart:
> Could someone show me some simple example code for incress+egress
> shaping for ppp0 (for a router with clients at eth0)?
Maybe my script will do: http://www.metamorpher.de/ipshape/

I don''t know about ''simple'', but I got a script
designed for
routers in general which have to provide masquerading, port
forwarding and traffic shaping for several clients in the LAN.
Even if it looks a bit complicated here and there, I think I got
it well documented, though. It looks pretty similar to what you
were trying to do.

I created this with the help of LARTC (Howto, Stef''s docum.org, and
of course this list) and it has grown a lot lately :-) You can
specify the IPs of your clients, and bandwidth will be shared
in a fair manner among them.

I use HTB, PRIO and SFQ to do that. It works well for me, but I''m
sure that there is still LOADS of stuff that can be improved.
I''m always open for suggestions :-)

Andreas
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Andreas Klauer wrote:
> Am Wednesday 28 April 2004 10:42 schrieb Christian Parpart:
> 
>>Could someone show me some simple example code for incress+egress
>>shaping for ppp0 (for a router with clients at eth0)?
> 
> 
> Maybe my script will do: http://www.metamorpher.de/ipshape/
> 
> I don''t know about ''simple'', but I got a script
designed for
> routers in general which have to provide masquerading, port
> forwarding and traffic shaping for several clients in the LAN.
> Even if it looks a bit complicated here and there, I think I got
> it well documented, though. It looks pretty similar to what you
> were trying to do.
> 
> I created this with the help of LARTC (Howto, Stef''s docum.org,
and
> of course this list) and it has grown a lot lately :-) You can
> specify the IPs of your clients, and bandwidth will be shared
> in a fair manner among them.
> 
> I use HTB, PRIO and SFQ to do that. It works well for me, but I''m
> sure that there is still LOADS of stuff that can be improved.
> I''m always open for suggestions :-)
Nice script - one thing I found was that HTB dequeued packets in pairs - 
with MTU 1500 and your 128kbit up this will hurt latency a bit.

The solution was to change from 1 to 0

#define HTB_HYSTERESIS 0 in net/sched/sch_htb.c

Andy.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Am Wednesday 05 May 2004 10:34 schrieb Andy Furniss:
> Andreas Klauer wrote:
> > Maybe my script will do: http://www.metamorpher.de/ipshape/
I renamed it to ''Fair NAT'' and moved it to 
http://www.metamorpher.de/fairnat/, because there already was another 
script called ipshape. I didn''t like the name anyway :-)
> Nice script - one thing I found was that HTB dequeued packets in pairs -
> with MTU 1500 and your 128kbit up this will hurt latency a bit.
>
> The solution was to change from 1 to 0
>
> #define HTB_HYSTERESIS 0 in net/sched/sch_htb.c
Thanks for the suggestion. I just recompiled the kernel - we''ll see if
I
notice any change. However, I don''t yet fully understand what
HYSTERESIS
actually does. There''s a FAQ on docum.org, but I still don''t
get it.
What does ''packets in pairs'' mean? Multiple packages at once
sounds to me
like burst.

I wish they would make such things available in kernel configuration menu,
with a proper explanation. If you look in the code, there is loads of stuff 
that can be customized in the kernel by changing defines directly, but you 
rarely can change those things via kernel config. :-(

Andreas
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Andreas Klauer wrote:
> Am Wednesday 05 May 2004 10:34 schrieb Andy Furniss:
> 
>>Andreas Klauer wrote:
>>
>>>Maybe my script will do: http://www.metamorpher.de/ipshape/
> 
> 
> I renamed it to ''Fair NAT'' and moved it to 
> http://www.metamorpher.de/fairnat/, because there already was another 
> script called ipshape. I didn''t like the name anyway :-)
> 
> 
>>Nice script - one thing I found was that HTB dequeued packets in pairs -
>>with MTU 1500 and your 128kbit up this will hurt latency a bit.
>>
>>The solution was to change from 1 to 0
>>
>>#define HTB_HYSTERESIS 0 in net/sched/sch_htb.c
> 
> 
> Thanks for the suggestion. I just recompiled the kernel - we''ll
see if I
> notice any change. However, I don''t yet fully understand what
HYSTERESIS
> actually does. There''s a FAQ on docum.org, but I still
don''t get it.
> What does ''packets in pairs'' mean? Multiple packages at
once sounds to me
> like burst.
YMMV of course - I have posted this here before.

I was using tcpdump a while back, to sus how (e)sfq worked. I had a very 
simple test setup, which just throttled bulk traffic to 51kbit my link 
is 256/512. I had burst set low and quantum to my MTU. Sniffing tcp 
after shaping I could see from the timestamps that the packets were 
being released in pairs - the rate was OK though. I changed timing from 
jiffies to cpu - no difference, I then remembered seeing the hysteresis 
page on stefs'' site and tried that and it fixed it.

I saw an improvement in my latency when my upstream was full - doing the 
maths, it behaves as expected now, ie. the worst case delay is my 
baseline latency + bitrate for my speed/mtu.

If your real (ie. not a cable modem) upstream is 128k then a 1500 byte 
packet is going to take about 80-90ms - so in theory when your up is 
full you should be able to notice the difference in max reading on ping. 
It will pull avg down aswell.

There are reasons it may make no difference for you though -

Your setup shares all traffic per IP - so if others are using their 
uprate you will queue anyway, I only do bulk per IP so in theory my 
interactive packets never queue (the rate/burst for my interactive class 
is way higher than the traffic should ever be - easy on a home setup, 
probably not so easy in real world)

I use MTU/quantum 1478 - which may or may not have caused the pairing in 
the first place - I didn''t test 1500.

I explicitly set low (c)bursts for bulk - I don''t know what the
defaults
will be for you not setting them - but I guess they should soon get used 
up anyway.

Andy.

> 
> I wish they would make such things available in kernel configuration menu,
> with a proper explanation. If you look in the code, there is loads of stuff
> that can be customized in the kernel by changing defines directly, but you 
> rarely can change those things via kernel config. :-(
> 
> Andreas
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> 

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/