LARTC - Mar 2005 - These tc commands used to work... what''s broken them?

I have a Gentoo Linux (kernel 2.6.11) server.  Several months ago, I
made a traffic shaping setup for my box (running a 2.4 kernel then)
that worked beautifully.  It gave high priority to SSH and RealAudio
traffic, and put HTTP downloading traffic on a lower prio so they could
only use what bandwidth was left.

However, I''ve only just realized that tc is no longer accepting the
commands I setup in a script to start it.  I don''t know when it stopped
working.

I can post the tcng script if anyone wants to see how I generated my tc 
commands, but I suspect it may not be relevant since tc is bombing out 
so early in the process.  Here''s what I get:

------------------------------------------------------------
> tc -s qdisc show dev eth0
qdisc pfifo_fast 0: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1   
 Sent 455120063 bytes 553706 pkt (dropped 0, overlimits 0 requeues 25)
 rate 0bit 0pps backlog 0b 0p requeues 25
> tc -s class show dev eth0
> tc qdisc add dev eth0 handle 1:0 root dsmark indices 8 default_index 0
RTNETLINK answers: Invalid argument
------------------------------------------------------------

Now this is such a basic command... and yet tc is already choking on   
it.  If I try something else, like this random example, then it works:

------------------------------------------------------------
> tc qdisc add dev eth0 handle 1:0 root cbq bandwidth 90000bps avpkt 1000
> tc -s class show dev eth0
class cbq 1: root rate 720000bit (bounded,isolated) prio no-transmit
 Sent 19872 bytes 36 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0
  borrowed 0 overactions 0 avgidle 11377 undertime 0
------------------------------------------------------------

Ok, so I said to myself "Aha, maybe when Gentoo upgraded its iproute2
package not too long ago, it didn''t compile in DSMARK support that the
old iproute2 had!" So I dug into the iproute2 package.  But as best I
can tell, the latest iproute2 (at least the Gentoo variant) doesn''t
even need the old TC_CONFIG_DIFFSERV=y directive in the Config file any
more.  It appears to compile dsmark support in in by default.  Just to
check, I tried this:
> strings /sbin/tc | grep dsmark
dsmark_qdisc_util
dsmark
dsmark: indices too short
dsmark: empty mask
dsmark: empty value
Usage: dsmark indices INDICES [ default_index DEFAULT_INDEX ] [
set_tc_index ]
Usage: ... dsmark [ mask MASK ] [ value VALUE ]
dsmark: default_index too short

So, it definitely looks like dsmark support is in there...

Next I thought "Maybe I forgot to compile in dsmark support when I
upgraded from a 2.4 to 2.6 kernel".  But no, that doesn''t seem to
be
the case:

Device Drivers ->
  Networking Support ->
    Networking Options ->
      QoS and/or fair queueing ->

        [*] QoS and/or fair queueing                               
        <M>   CBQ packet scheduler                                 
        <M>   HTB packet scheduler                                 
        <M>   HFSC packet scheduler                                
        <M>   The simplest PRIO pseudoscheduler                    
        <M>   RED queue                                            
        <M>   SFQ queue                                            
        <M>   TEQL queue                                           
        <M>   TBF queue                                            
        <M>   GRED queue                                           
        <M>   Diffserv field marker                                
        <M>   Network emulator                                     
        <M>   Ingress Qdisc                                        
        [*]   QoS support                                          
        [*]     Rate estimator                                     
        [*]   Packet classifier API                                
        <M>     TC index classifier                                
        <M>     Routing table based classifier                     
        <M> Firewall based classifier                              
        <M> U32 classifier                                         
        [ ]   U32 classifier performance counters                  
        [ ] classify input device (slows things u32/fw)            
        [ ] Use nfmark as a key in U32 classifier                  
        <M> Special RSVP classifier                                
        <M> Special RSVP classifier for IPv6                       
        [ ] Packet ACTION                                          
        [*] Traffic policing (needed for in/egress)                

So I''m stumped.  Can anyone see anything obvious that would cause tc to
be choking on commands that used to work fine?

I''ll be happy to provide any further information (tcng script, full
list of tc commands, etc.) if it''ll help.  I hope someone has an idea,
because I don''t know where else to look.

George Adams wrote:
> I have a Gentoo Linux (kernel 2.6.11) server.  Several months ago, I
> made a traffic shaping setup for my box (running a 2.4 kernel then)
> that worked beautifully.  It gave high priority to SSH and RealAudio
> traffic, and put HTTP downloading traffic on a lower prio so they could
> only use what bandwidth was left.
> 
I''m not familiar with Gentoo or dsmark so I can only give you a very
general answer. Someone else might know more.

Try compiling iproute2 from http://developer.osdl.org/dev/iproute2/. You
 don''t necessarily need to ''make install''; just test
by running the new
tc binary in the source tree.

Try compiling a vanilla kernel from http://www.kernel.org/, if you
aren''t running one already.

Try various combinations and see if you can find one that works. You
want to determine if this is a Gentoo-specific problem. If it is, you
should report your bug to the respective Gentoo package maintainer(s).
If it isn''t a Gentoo problem, you can report it upstream to whomever it
concerns.

-Corey

* George Adams <20050325220314.73F5D32AB1D@www.davidwhite.org> 2005-03-25
17:03
> > tc qdisc add dev eth0 handle 1:0 root dsmark indices 8 default_index 0
> 
> RTNETLINK answers: Invalid argument
This has been fixed in the latest version which is

 
http://developer.osdl.org/shemminger/iproute2/download/iproute2-2.6.11-050330.tar.gz

The bug was pretty old but was hidden until some recent changes.

George Adams wrote:
> I have a Gentoo Linux (kernel 2.6.11) server.  Several months ago, I
> made a traffic shaping setup for my box (running a 2.4 kernel then)
> that worked beautifully.  It gave high priority to SSH and RealAudio
> traffic, and put HTTP downloading traffic on a lower prio so they could
> only use what bandwidth was left.
> 
> However, I''ve only just realized that tc is no longer accepting
the
> commands I setup in a script to start it.  I don''t know when it
stopped
> working.
> 
> I can post the tcng script if anyone wants to see how I generated my tc 
> commands, but I suspect it may not be relevant since tc is bombing out 
> so early in the process.  Here''s what I get:
> 
> ------------------------------------------------------------
> 
>>tc -s qdisc show dev eth0
> 
> 
> qdisc pfifo_fast 0: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1   
>  Sent 455120063 bytes 553706 pkt (dropped 0, overlimits 0 requeues 25)
>  rate 0bit 0pps backlog 0b 0p requeues 25
> 
> 
>>tc -s class show dev eth0
> 
> 
>>tc qdisc add dev eth0 handle 1:0 root dsmark indices 8 default_index 0
That works for me using the latest iproute2 and 2.6.12-rc1 I think you 
just hit unlucky with your version/kernel and it''s been fixed now.

http://developer.osdl.org/dev/iproute2/download/iproute2-2.6.11-050330.tar.gz

Andy.