LARTC - Apr 2007 - Multiple bands with equal priority ?

I''m trying to build a wan latency test environment, where packets
from different "remote" locations get delayed by different amounts
of time, depending on which remote location we''re pretending they
are from.

Currently, I''m doing this using the ''prio'' qdisc to
obtain multiple
bands, and hanging a different netem qdisc off each of the branches
to delay packets, like this (assuming two remote locations):

# create three bands, and place all default traffic in the first one:
tc qdisc add dev eth0 root handle 1: \
      prio bands 3 priomap 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

# the second and third bands of the prio above will delay packets
# by 50ms for one simulated remote location, and by 70ms for the other:
tc qdisc add dev eth0 parent 1:2 handle 20: netem delay 50ms
tc qdisc add dev eth0 parent 1:3 handle 30: netem delay 70ms

# use filters to place ''remote'' traffic into the appropriate
band:
tc filter add dev eth0 protocol ip parent 1:0 \
      prio 2 <...match for first remote location...> flowid 10:2
tc filter add dev eth0 protocol ip parent 1:0 \
      prio 3 <...match for second remote location...> flowid 10:3

This works great, but I can only really test one remote location at a
time, because otherwise traffic sent to the second band will always
starve out traffic sent to the third band. I actually don''t mind
default traffic having priority over that from my two ''remote''
locations... :)

Anyway, I''m looking for a way to allow packets from the two remote
locations to compete for bandwidth on equal footing (after the
appropriate delay has been applied, of course). So, instead of
a prio multiband qdisc, I''d be interested in a round-robin one.

Can I accomplish this with CBQ ? What would the tc commands have
to look like -- I''m getting slightly confused by the split/defmap
syntax, and by trying to figure out when it''s a clas vs a qdisc
I''m
supposed to be dealing with... :(

I guess I should be looking at using the WRR qdisc, but I''d like to
try everything else I can before falling through to adding an
out-of-tree kernel module and patching tc...

Thanks,
Gabriel

Gabriel Somlo wrote:
> I''m trying to build a wan latency test environment, where packets
> from different "remote" locations get delayed by different
amounts
> of time, depending on which remote location we''re pretending they
> are from.
> 
> Currently, I''m doing this using the ''prio'' qdisc
to obtain multiple
> bands, and hanging a different netem qdisc off each of the branches
> to delay packets, like this (assuming two remote locations):
> 
> # create three bands, and place all default traffic in the first one:
> tc qdisc add dev eth0 root handle 1: \
>      prio bands 3 priomap 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> 
> # the second and third bands of the prio above will delay packets
> # by 50ms for one simulated remote location, and by 70ms for the other:
> tc qdisc add dev eth0 parent 1:2 handle 20: netem delay 50ms
> tc qdisc add dev eth0 parent 1:3 handle 30: netem delay 70ms
> 
> # use filters to place ''remote'' traffic into the
appropriate band:
> tc filter add dev eth0 protocol ip parent 1:0 \
>      prio 2 <...match for first remote location...> flowid 10:2
> tc filter add dev eth0 protocol ip parent 1:0 \
>      prio 3 <...match for second remote location...> flowid 10:3
> 
> This works great, but I can only really test one remote location at a
> time, because otherwise traffic sent to the second band will always
> starve out traffic sent to the third band. I actually don''t mind
> default traffic having priority over that from my two
''remote''
> locations... :)
> 
> Anyway, I''m looking for a way to allow packets from the two remote
> locations to compete for bandwidth on equal footing (after the
> appropriate delay has been applied, of course). So, instead of
> a prio multiband qdisc, I''d be interested in a round-robin one.
> 
> Can I accomplish this with CBQ ? What would the tc commands have
> to look like -- I''m getting slightly confused by the split/defmap
> syntax, and by trying to figure out when it''s a clas vs a qdisc
I''m
> supposed to be dealing with... :(
> 
> I guess I should be looking at using the WRR qdisc, but I''d like
to
> try everything else I can before falling through to adding an
> out-of-tree kernel module and patching tc...
Hmm I didn''t think that prio would behave this way unless the eth was 
backlogged - but then I haven''t tried this setup.

I would use ifb  - if the remote wans are real and all you need to do is 
add latency then just netem on roots, if you need to simulate the low 
bandwidth as well, then there is an example of using tbf + netem on the 
netem site - still use an ifb per class.

Andy.