1.I have commands to make filters like your old but have no this: "ht 2:$table1: ht 3:$table2:" what is it? 2. What was your problem? I''m new to this list and dont have your previous email.> Old command: > tc filter add dev $dev protocol ip parent 12: u32 ht 2:$table1: ht > 3:$table2: match ip dst $ip/32 flowid 12:$classid > > New command: > tc filter add dev $dev protocol ip parent 12: prio 5 u32 ht 2:$table1: > ht 3:$table2: match ip dst $ip/32 flowid 12:$classid > > Who would have thought that a prio would cause that much trouble? I > mean, I know in the documentation it says that it is required for CLASS > commands, but since there''s nothing on filters, I had no clue. And > usually if a command requires a parameter, it will error immediately and > not wait for 2045 entries. Also, why the hell would the priority cause a > problem like that to begin with?! > > Weird. Anyways, THANKS EVERYONE! (gypsy, Catalin, and Ed especially) It > all works, it''s very resource friendly, and all is good again (and I get > to keep my job)! The processor is only about 20% in use now. I''m very > grateful for the hashing filter suggestion from Ed. Thanks a ton. I will > be posting a How To shortly so others can do the same if they ever need > to. Thanks again! > > Adam Towarnyckyj > > -----Original Message----- > From: lartc-admin@mailman.ds9a.nl [mailto:lartc-admin@mailman.ds9a.nl] > On Behalf Of gypsy > Sent: Thursday, July 15, 2004 8:37 PM > To: adamt@commspeed.net; LARTC > Subject: Re: [LARTC] TC Hashing Filters > > Adam Towarnyckyj wrote: > > Any tc filter command I try to > > add from here on out gives me the same error even if I try it manually > > instead of using the script. > > Adam, > > That is a killer. > > Please read the following and then, when we both have the same > information, I''m going to try again to assist you. > > HINT TO READERS: I hope someone else will help us both because I have > obligations that I''m stealing time from that I can ill afford. > > There is a LARTC mailing list thread dated (about) 24 thru 26 June 2003 > between Trevor Warren and Michael Ulitskiy whose Subject is "u32 > clarification...limits on 2000>???" > > Please use your favorite method to find it. Note that there may be > kernel issues not mentioned by them; kernels change. > > I''d like to suggest that you see if anyone involved in that thread will > send you a testing script; perhaps you could find a way to start with a > working setup and then apply minor changes until either it breaks or it > suits you. > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hey all, I had asked a question a little while ago about CPU usage being outrageous while using tc and a cbq qdisc. Ed was very kind and offered a few suggestions. One of these was to look at hashing. So after pounding away at it for about a week, I have a general understanding of how it works and have tried to implement it. Unfortunately, and quite obviously since I''m posting here, it does not work for me. The Problem: When implemented, and all the tc commands are entered, I get no errors or anything for that matter. It looks like everything went in smoothly. I do a show and sure enough, it is all there and looks good. However, when I have a computer hooked up on one side of the limiter and the other side hooked up to the internet, I do a download test and get the average 35Mbps that I normally see from our office. The Situation: The first step in troubleshooting this problem involved finding out if tc even limited at all. So I set up a simple class with a 128Kbit rate and cbq root qdisc like I had originally and all worked well. The download test showed 128Kbit/s like it should. I was happy that this part works. It seems that when I install the hash tables and assign it to hash based on the last octet in the IP address, it just allows traffic to pass through without limiting. This is where I run into the trouble. The Commands: Here is what I originally started out with based on the Hashing Filter How To: # Create root qdisc tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit avpkt 1000 # Create a "transit class" tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 200Mbit rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 # Create hash table attached to transit class tc filter add dev eth1 parent 1:0 handle 2: protocol ip u32 divisor 256 # Create filter to hash out last octet and link to hash table 2: tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht 800:: match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2: # Create class for 128Kbit limit tc class add dev eth1 parent 1:0 classid 1:2 cbq bandwidth 200Mbit rate 128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded # Create filter for IP I''m limiting tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c match ip src 216.19.49.140 flowid 1:2 Troubleshooting: There are a few things I''d like to point out. First off, I''m only working with one IP address at the moment and the traffic is coming from only one Class C at the moment. I''ve even tried hard setting the hash ip src to 216.19.49.140/32 and that doesn''t work. Second, I''ve tried attaching each of these to different parents. I tried attaching the hash table to the root qdisc as well as the transit class. I''ve tried the same for the limiting class and the ip filter. None of this works. Third, I''ve also tried it without the transit class at all and just doing everything direct off the root qdisc. Not only have I used the Hashing Filter How To but I have also used some suggestions from Gideon who posted to this list in the past. (http://mailman.ds9a.nl/pipermail/lartc/2003q2/008516.html). I tried htb as well and this qdisc didn''t work either. Also, I''ve changed the src to dst and I originally started out using the hash ip src as our entire network of 216.19.0.0/18. If anyone has an understanding of hashing filters, please PLEASE let me know what I''m doing wrong. I''ve been working on this for three days straight now and in the meantime, our bandwidth limiting is not running which is INCREDIBLY important to my boss. I currently have just a straight list of 5000 users that I do not want to stray too far from (the iptables project Ed referred to requires a complete overhaul of my current implementation). I''m pulling my hair out because according to my understanding of it and based on what others have done this SHOULD be working..... Thank you very much for at least taking the time to read this. I truly appreciate it. I''m also hoping that this will spark a thread for future people who run into the same trouble I am having. Thanks. Adam Towarnyckyj _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> # Create filter to hash out last octet and link to hash table 2: > tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht >800:: match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2:I think this needs to be attached to the root. parent 1:0 Adam Towarnyckyj (adamt@commspeed.net) wrote:> >Hey all, > I had asked a question a little while ago about CPU usage being >outrageous while using tc and a cbq qdisc. Ed was very kind and offered >a few suggestions. One of these was to look at hashing. So after >pounding away at it for about a week, I have a general understanding of >how it works and have tried to implement it. Unfortunately, and quite >obviously since I''m posting here, it does not work for me. > >The Problem: > When implemented, and all the tc commands are entered, I get no >errors or anything for that matter. It looks like everything went in >smoothly. I do a show and sure enough, it is all there and looks good. >However, when I have a computer hooked up on one side of the limiter and >the other side hooked up to the internet, I do a download test and get >the average 35Mbps that I normally see from our office. > >The Situation: > The first step in troubleshooting this problem involved finding >out if tc even limited at all. So I set up a simple class with a 128Kbit >rate and cbq root qdisc like I had originally and all worked well. The >download test showed 128Kbit/s like it should. I was happy that this >part works. > It seems that when I install the hash tables and assign it to >hash based on the last octet in the IP address, it just allows traffic >to pass through without limiting. This is where I run into the trouble. > >The Commands: > Here is what I originally started out with based on the Hashing >Filter How To: > > # Create root qdisc > tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit >avpkt 1000 > > # Create a "transit class" > tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth >200Mbit rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt >1000 > > # Create hash table attached to transit class > tc filter add dev eth1 parent 1:0 handle 2: protocol ip u32 >divisor 256 > > # Create filter to hash out last octet and link to hash table 2: > tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht >800:: match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2: > > # Create class for 128Kbit limit > tc class add dev eth1 parent 1:0 classid 1:2 cbq bandwidth >200Mbit rate 128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded > > # Create filter for IP I''m limiting > tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c >match ip src 216.19.49.140 flowid 1:2 > >Troubleshooting: > There are a few things I''d like to point out. First off, I''m >only working with one IP address at the moment and the traffic is coming >from only one Class C at the moment. I''ve even tried hard setting the >hash ip src to 216.19.49.140/32 and that doesn''t work. Second, I''ve >tried attaching each of these to different parents. I tried attaching >the hash table to the root qdisc as well as the transit class. I''ve >tried the same for the limiting class and the ip filter. None of this >works. Third, I''ve also tried it without the transit class at all and >just doing everything direct off the root qdisc. Not only have I used >the Hashing Filter How To but I have also used some suggestions from >Gideon who posted to this list in the past. >(http://mailman.ds9a.nl/pipermail/lartc/2003q2/008516.html). I tried htb >as well and this qdisc didn''t work either. Also, I''ve changed the src to >dst and I originally started out using the hash ip src as our entire >network of 216.19.0.0/18. > If anyone has an understanding of hashing filters, please PLEASE >let me know what I''m doing wrong. I''ve been working on this for three >days straight now and in the meantime, our bandwidth limiting is not >running which is INCREDIBLY important to my boss. I currently have just >a straight list of 5000 users that I do not want to stray too far from >(the iptables project Ed referred to requires a complete overhaul of my >current implementation). I''m pulling my hair out because according to my >understanding of it and based on what others have done this SHOULD be >working..... > Thank you very much for at least taking the time to read this. I >truly appreciate it. I''m also hoping that this will spark a thread for >future people who run into the same trouble I am having. Thanks. > >Adam Towarnyckyj > > >_______________________________________________ >LARTC mailing list / LARTC@mailman.ds9a.nl >http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >-- When dealing with a slow pipe, never underestimate the throughput of the postal system. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
zoop@lone.ath.cx wrote:>> # Create filter to hash out last octet and link to hash table 2: >> tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht >>800:: match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link2:> > >I think this needs to be attached to the root. >parent 1:0Catalin BOIE (util@deuroconsult.ro) wrote:>> # Create root qdisc >> tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbitavpkt 1000>> # Create a "transit class" >> tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth200Mbit>> rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 >> # Create hash table attached to transit class >> tc filter add dev eth1 parent 1:0 handle 2: protocol ip u32divisor 256>> # Create filter to hash out last octet and link to hash table 2: >> tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht >> 800:: match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link2:>> # Create class for 128Kbit limit >> tc class add dev eth1 parent 1:0 classid 1:2 cbq bandwidth > >I think here is parent 1:1, right? > >> 200Mbit rate 128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded >> # Create filter for IP I''m limiting >> tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c>> match ip src 216.19.49.140 flowid 1:2I have tried both of your suggestions and it is still allowing all traffic to pass through. Thank you for the help though. Here is what I have now: # Create root qdisc tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit avpkt 1000 # Create transit class tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 200Mbit rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 # Create hash table and attach to transit class tc filter add dev eth1 parent 1:1 handle 2: protocol ip u32 divisor 256 # Create filter to hash out last octet and link to hash table 2: tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 800:: match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2: # Create class for 128Kbit limit tc class add dev eth1 parent 1:1 classid 1:2 cbq bandwidth 200Mbit rate 128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded # Create filter for IP I''m limiting tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c match ip src 216.19.49.140 flowid 1:2 If anyone can still see anything I''m doing wrong, I''d appreciate it if you pointed it out. I''ve tried many different scenarios with different parents and they still don''t work. And as for the question earlier about if I was doing this for an ISP, yes I am. If you would like some help, and if I can get this to work, I plan on writing a How To and posting it to a website so people can easily set this up themselves. I find the lack of documentation on the subject quite frustrating at times. Thanks all for your help. Adam Towarnyckyj _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
----Original Message----># Create root qdisc >tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit avpkt 1000 > ># Create transit class >tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 200Mbit rate >200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 > ># Create hash table and attach to transit class >tc filter add dev eth1 parent 1:1 handle 2: protocol ip u32 divisor 256 > ># Create filter to hash out last octet and link to hash table 2: >tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 800:: match >ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2: > ># Create class for 128Kbit limit >tc class add dev eth1 parent 1:1 classid 1:2 cbq bandwidth 200Mbit rate >128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded > ># Create filter for IP I''m limiting >tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c match >ip src 216.19.49.140 flowid 1:2I messed with this a bit more before finally just giving up for today. I looked at Gideon''s example he posted to this list in the past (http://mailman.ds9a.nl/pipermail/lartc/2003q2/008516.html) and copied it character by character except I changed the ethernet device it was attached to and the IP addresses he was using. Here''s what I came up with: tc qdisc add dev eth1 root handle 1: htb tc class add dev eth1 parent 1: classid 1:2 htb rate 100MBit ceil 100MBit burst 0Kbit tc filter add dev eth1 parent 1:2 handle 2: protocol ip u32 divisor 256 tc filter add dev eth1 protocol ip parent 1: u32 match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2: tc class add dev eth1 parent 1:2 classid 1:3 htb rate 64Kbit ceil 64Kbit burst 0Kbit tc filter add dev eth1 protocol ip parent 1:2 u32 ht 2:8c: match ip src 216.19.49.140 flowid 1:3 I even tried changing the src to dst and this still does not work. Gideon, if you are out there somewhere, I''d appreciate it if you could tell me if this actually worked for you. This is the most frustrating project I have ever had to deal with because it makes so much sense and in theory it looks like it should be working just fine. Anyways, thanks for everyone''s help. If anyone has any other ideas, I welcome them. Adam Towarnyckyj _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Adam Towarnyckyj (adamt@commspeed.net) wrote:> >----Original Message---- >># Create root qdisc >>tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit avpkt 1000 >> >># Create transit class >>tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 200Mbit rate >>200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 >> >># Create hash table and attach to transit class >>tc filter add dev eth1 parent 1:1 handle 2: protocol ip u32 divisor 256 >> >># Create filter to hash out last octet and link to hash table 2: >>tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 800:: match >>ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2: >> >># Create class for 128Kbit limit >>tc class add dev eth1 parent 1:1 classid 1:2 cbq bandwidth 200Mbit rate >>128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded >> >># Create filter for IP I''m limiting >>tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c match >>ip src 216.19.49.140 flowid 1:2 > >I messed with this a bit more before finally just giving up for today. I >looked at Gideon''s example he posted to this list in the past >(http://mailman.ds9a.nl/pipermail/lartc/2003q2/008516.html) and copied >it character by character except I changed the ethernet device it was >attached to and the IP addresses he was using. Here''s what I came up >with: > >tc qdisc add dev eth1 root handle 1: htb > >tc class add dev eth1 parent 1: classid 1:2 htb rate 100MBit ceil >100MBit burst 0Kbit >Here the filter parents should be root not 1:2. I think in the howto it says that all filters should be attached to root. I''ll get my box up that I have doing this to give you a working example.>tc filter add dev eth1 parent 1:2 handle 2: protocol ip u32 divisor 256 > >tc filter add dev eth1 protocol ip parent 1: u32 match ip src >216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2: > >tc class add dev eth1 parent 1:2 classid 1:3 htb rate 64Kbit ceil 64Kbit >burst 0Kbit > >tc filter add dev eth1 protocol ip parent 1:2 u32 ht 2:8c: match ip src >216.19.49.140 flowid 1:3 > >I even tried changing the src to dst and this still does not work. >Gideon, if you are out there somewhere, I''d appreciate it if you could >tell me if this actually worked for you. This is the most frustrating >project I have ever had to deal with because it makes so much sense and >in theory it looks like it should be working just fine. Anyways, thanks >for everyone''s help. If anyone has any other ideas, I welcome them. > >Adam Towarnyckyj > >_______________________________________________ >LARTC mailing list / LARTC@mailman.ds9a.nl >http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >-- When dealing with a slow pipe, never underestimate the throughput of the postal system. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
The reason I asked about if you were doing this for an ISP is that I worte an app to track IP''s and the bandwidths associated with them, and it will generate the TC commands for you. It uses HTB though. Adam Towarnyckyj (adamt@commspeed.net) wrote:> >zoop@lone.ath.cx wrote: >>> # Create filter to hash out last octet and link to hash table 2: >>> tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht >>>800:: match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link >2: >> >> >>I think this needs to be attached to the root. >>parent 1:0 > >Catalin BOIE (util@deuroconsult.ro) wrote: >>> # Create root qdisc >>> tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit >avpkt 1000 >>> # Create a "transit class" >>> tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth >200Mbit >>> rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 >>> # Create hash table attached to transit class >>> tc filter add dev eth1 parent 1:0 handle 2: protocol ip u32 >divisor 256 >>> # Create filter to hash out last octet and link to hash table 2: >>> tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht >>> 800:: match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link >2: >>> # Create class for 128Kbit limit >>> tc class add dev eth1 parent 1:0 classid 1:2 cbq bandwidth >> >>I think here is parent 1:1, right? >> >>> 200Mbit rate 128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded >>> # Create filter for IP I''m limiting >>> tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c > >>> match ip src 216.19.49.140 flowid 1:2 > > > I have tried both of your suggestions and it is still allowing >all traffic to pass through. Thank you for the help though. Here is what >I have now: > ># Create root qdisc >tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit avpkt 1000 > ># Create transit class >tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 200Mbit rate >200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 > ># Create hash table and attach to transit class >tc filter add dev eth1 parent 1:1 handle 2: protocol ip u32 divisor 256 > ># Create filter to hash out last octet and link to hash table 2: >tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 800:: match >ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2: > ># Create class for 128Kbit limit >tc class add dev eth1 parent 1:1 classid 1:2 cbq bandwidth 200Mbit rate >128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded > ># Create filter for IP I''m limiting >tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c match >ip src 216.19.49.140 flowid 1:2 > > If anyone can still see anything I''m doing wrong, I''d appreciate >it if you pointed it out. I''ve tried many different scenarios with >different parents and they still don''t work. And as for the question >earlier about if I was doing this for an ISP, yes I am. If you would >like some help, and if I can get this to work, I plan on writing a How >To and posting it to a website so people can easily set this up >themselves. I find the lack of documentation on the subject quite >frustrating at times. Thanks all for your help. > >Adam Towarnyckyj > >_______________________________________________ >LARTC mailing list / LARTC@mailman.ds9a.nl >http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >-- When dealing with a slow pipe, never underestimate the throughput of the postal system. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Where can I find that app? I would like to evaluate it for my site. Regards, Walt Wyndroski ----- Original Message ----- From: <zoop@lone.ath.cx> To: "Adam Towarnyckyj" <adamt@commspeed.net>; <lartc@mailman.ds9a.nl> Sent: Tuesday, July 06, 2004 3:16 PM Subject: RE: [LARTC] TC Hashing Filters> The reason I asked about if you were doing this for an ISP is that I wortean app to> track IP''s and the bandwidths associated with them, and it will generatethe TC> commands for you. It uses HTB though. > > Adam Towarnyckyj (adamt@commspeed.net) wrote: > > > >zoop@lone.ath.cx wrote: > >>> # Create filter to hash out last octet and link to hash table 2: > >>> tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht > >>>800:: match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link > >2: > >> > >> > >>I think this needs to be attached to the root. > >>parent 1:0 > > > >Catalin BOIE (util@deuroconsult.ro) wrote: > >>> # Create root qdisc > >>> tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit > >avpkt 1000 > >>> # Create a "transit class" > >>> tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth > >200Mbit > >>> rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 > >>> # Create hash table attached to transit class > >>> tc filter add dev eth1 parent 1:0 handle 2: protocol ip u32 > >divisor 256 > >>> # Create filter to hash out last octet and link to hash table 2: > >>> tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht > >>> 800:: match ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link > >2: > >>> # Create class for 128Kbit limit > >>> tc class add dev eth1 parent 1:0 classid 1:2 cbq bandwidth > >> > >>I think here is parent 1:1, right? > >> > >>> 200Mbit rate 128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded > >>> # Create filter for IP I''m limiting > >>> tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c > > > >>> match ip src 216.19.49.140 flowid 1:2 > > > > > > I have tried both of your suggestions and it is still allowing > >all traffic to pass through. Thank you for the help though. Here is what > >I have now: > > > ># Create root qdisc > >tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit avpkt 1000 > > > ># Create transit class > >tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 200Mbit rate > >200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 > > > ># Create hash table and attach to transit class > >tc filter add dev eth1 parent 1:1 handle 2: protocol ip u32 divisor 256 > > > ># Create filter to hash out last octet and link to hash table 2: > >tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 800:: match > >ip src 216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2: > > > ># Create class for 128Kbit limit > >tc class add dev eth1 parent 1:1 classid 1:2 cbq bandwidth 200Mbit rate > >128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded > > > ># Create filter for IP I''m limiting > >tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c match > >ip src 216.19.49.140 flowid 1:2 > > > > If anyone can still see anything I''m doing wrong, I''d appreciate > >it if you pointed it out. I''ve tried many different scenarios with > >different parents and they still don''t work. And as for the question > >earlier about if I was doing this for an ISP, yes I am. If you would > >like some help, and if I can get this to work, I plan on writing a How > >To and posting it to a website so people can easily set this up > >themselves. I find the lack of documentation on the subject quite > >frustrating at times. Thanks all for your help. > > > >Adam Towarnyckyj > > > >_______________________________________________ > >LARTC mailing list / LARTC@mailman.ds9a.nl > >http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ > > > > -- > When dealing with a slow pipe, never underestimate the throughput of thepostal system.> > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Ok, I tried this: tc qdisc add dev eth1 root handle 1: cbq bandwidth 200Mbit avpkt 1000 tc class add dev eth1 parent 1: classid 1:2 cbq bandwidth 200Mbit rate 200MBit allot 1514 weight 2Mbit prio 8 maxburst 20 avpkt 1000 tc filter add dev eth1 parent 1: handle 2: protocol ip u32 divisor 256 tc filter add dev eth1 protocol ip parent 1: u32 match ip src 216.19.49.140/32 hashkey mask 0x000000ff at 12 link 2: tc class add dev eth1 parent 1:2 classid 1:3 cbq bandwidth 200Mbit rate 128Kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded tc filter add dev eth1 protocol ip parent 1: u32 ht 2:8c: match ip src 216.19.49.140 flowid 1:3 As well as changing the last class parent to 1: Neither of these worked. As for the app, could I take a look at it? I''m sure it could be easily modified to use CBQ and if not, I could modify the box the use HTB. I really don''t mind either way. I was having a problem patching for HTB and that''s why I''m not currently using it. My boss and I wrote a perl script to pull information from a MySql database (rate and customer IP) and plug it in to TC and that''s what I was planning on doing this time around only with a few modifications for the hashing tables. I won''t start on that though until I can get this up and running. Anyways, thanks for the help. Adam Towarnyckyj -----Original Message----- From: zoop@lone.ath.cx [mailto:zoop@lone.ath.cx] Sent: Tuesday, July 06, 2004 12:15 PM To: Adam Towarnyckyj; lartc@mailman.ds9a.nl Subject: RE: [LARTC] TC Hashing Filters .... Here the filter parents should be root not 1:2. I think in the howto it says that all filters should be attached to root. I''ll get my box up that I have doing this to give you a working example.>tc filter add dev eth1 parent 1:2 handle 2: protocol ip u32 divisor 256 > >tc filter add dev eth1 protocol ip parent 1: u32 match ip src >216.19.49.0/24 hashkey mask 0x000000ff at 12 link 2: > >tc class add dev eth1 parent 1:2 classid 1:3 htb rate 64Kbit ceil64Kbit>burst 0Kbit > >tc filter add dev eth1 protocol ip parent 1:2 u32 ht 2:8c: match ip src >216.19.49.140 flowid 1:3 > >I even tried changing the src to dst and this still does not work. >Gideon, if you are out there somewhere, I''d appreciate it if you could >tell me if this actually worked for you. This is the most frustrating >project I have ever had to deal with because it makes so much sense and >in theory it looks like it should be working just fine. Anyways, thanks >for everyone''s help. If anyone has any other ideas, I welcome them. > >Adam Towarnyckyj_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Gideon was an incredible help with all of this. He solved my little problem I was having. It was such a simple solution and I completely missed it. You see, when I changed the src to dst (which is what I''m trying to accomplish because I''m shaping the download rates of our customers, not upload rates) I forgot that in the IP Header, bit 12 is the source address. Since I was shaping incoming traffic, I want the destination IP. Gideon was so kind to point this out to me. I knew it was something very small and obvious... So, here are the completed commands that work great for anyone who is attempting to do this themselves. I will also post to this list with this same subject the link to the How To as soon as I get it created. # Delete current qdisc wiping the slate clean tc qdisc del dev eth1 root # Create root qdisc tc qdisc add dev eth1 root handle 1: cbq bandwidth 200Mbit avpkt 1000 # Create "transit class" tc class add dev eth1 parent 1: classid 1:2 cbq bandwidth 200Mbit rate 200MBit allot 1514 weight 2Mbit prio 8 maxburst 20 avpkt 1000 # Create hash tables (256) tc filter add dev eth1 parent 1: handle 2: protocol ip u32 divisor 256 # Create hash filter to separate traffic going TO 216.19.49.0/24 using the last octet tc filter add dev eth1 protocol ip parent 1: u32 match ip dst 216.19.49.0/24 hashkey mask 0x000000ff at 16 link 2: # Create the 128Kbit class for limiting tc class add dev eth1 parent 1: classid 1:3 cbq bandwidth 200Mbit rate 128Kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded # Filter per IP into the hash tables tc filter add dev eth1 protocol ip parent 1: u32 ht 2:8c: match ip dst 216.19.49.140 flowid 1:3 (Note: The last filter will need to be done for each IP you want to limit. Also note that you can create multiple classes for different rates you would like to limit. You''ll just need to point the last filter''s flowid to that class) Thanks, once again, to everyone for their help and the pointing out of my obvious mistakes. If anyone has any questions about how to do this or anything, please feel free to ask. I''ll answer to the best of my abilities. Adam Towarnyckyj _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>If anyone has any questions about how to do this or >anything, please feel free to ask. I''ll answer to the best of my >abilities. > >I have one. How much faster does it go now? _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hey all, So I got the script to run and populate everything. I watch as the script uses the tc commands to add filters for each IP into their appropriate tables. After 2045 entries, it starts to give me a "File Exists" error. I''ve done extensive testing on the script and everything else to come to this same result. I also looked at how many entries were going into individual tables. It seems there are only 15-20 per table and this isn''t a lot. So my question is does the hash filter have a limit on the number of entries? I wouldn''t believe so but I keep running into this problem. If this is the case, I guess I''ll be looking for another alternative. Thanks! Adam Towarnyckyj _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Adam Towarnyckyj wrote:> > Hey all, > So I got the script to run and populate everything. I watch as > the script uses the tc commands to add filters for each IP into their > appropriate tables. After 2045 entries, it starts to give me a "File > Exists" error. I''ve done extensive testing on the script and everythingCOMMENT: Why is it that everyone who has a bad script does not post (even a snip of) it? The limit is 65536. If you don''t get the significance of that, it means the numbers are hex and you''re trying to make ''em be decimal. Go read docum.org. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
First off, no need to be rude. I read the documentation; I just missed
the number limit.
Second, I know it''s in hex and that''s what I''m using.
Third, the script is rather large. I didn''t want to post even a snippet
because it is so big. But, if that''s what you require in order to give
me some advice, then here it is:
Action("$tc qdisc del dev $dev root");
Action("$tc qdisc add dev $dev root handle 1:0 cbq bandwidth 200mbit
avpkt 1000");
# Create ''transit class'', tc hash tables, and hash filter
Action("$tc class add dev $dev parent 1: classid 1:2 cbq bandwidth
200Mbit rate 200MBit allot 1514 weight 2Mbit prio 8 maxburst 20 avpkt
1000");
Action("$tc filter add dev $dev parent 1: handle 2: protocol ip u32
divisor 256");
Action("$tc filter add dev $dev protocol ip parent 1: u32 match ip dst
0.0.0.0/0 hashkey mask 0x000000ff at 16 link 2:");
# Create classes for rate groups
@RATES = SelectSQL("SELECT dsrate FROM dsrate");
my $classid = 3;
foreach $dsrate (@RATES) {
$ds = $$dsrate{dsrate};
if ($ds == "0" || $ds == "1") {
next;
}
Action("$tc class add dev eth1 parent 1: classid 1:$classid cbq
bandwidth 200Mbit rate $$dsrate{dsrate}Kbit allot 1514 prio 5 maxburst
20 avpkt 1000 bounded");
$rates{$ds} = $classid;
$classid++;
}
# Get our list of accounts
@MODEMS = SelectSQL("SELECT mid, dsrate FROM modems");
$z = 0;
# Figure out account IPs and put ''em in!
foreach $modem (@MODEMS) {
if ($$modem{dsrate} == "0" || $$modem{dsrate} ==
"1") {
next;
}
my @COMPUTERS = SelectSQL("SELECT ipid FROM computers WHERE
mid=''$$modem{mid}''");
foreach $computer (@COMPUTERS) {
my $ip = SelectSingleSQL("SELECT ipaddr FROM ips WHERE
ipid=''$$computer{ipid}''");
@octets = split(/\./,$ip);
$table = $octets[3];
$table = sprintf("%X", $table);
$classid = $rates{$$modem{dsrate}};
Action("$tc filter add dev $dev protocol ip parent 1:
u32 ht 2:$table: match ip dst $ip flowid 1:$classid");
$z++;
print "$z\n";
}
}
"Action" is a sub that performs a system action and reports errors to
another sub.
"SelectSQL" is a sub that performs a sql query to our database.
COMMENT: Why is it whenever anyone new posts a question to a list even
after reading through the documentation, someone always has to jump down
his throat instead of being helpful and kind? This is a mailing list
which was created so people can ask questions and get a helpful response
from a community. I have been nothing but nice to everyone here and I am
really grateful for all the help that has been provided so far. Instead
of barking at people, simply stating that you require a snippet of code
and providing an answer would be much more helpful and wouldn''t cause
people like me to bark back which is surely going to cause an argument
between us.
Thanks again!
Adam Towarnyckyj
-----Original Message-----
From: gypsy [mailto:gypsy@iswest.com]
Sent: Monday, July 12, 2004 6:24 PM
To: adamt@commspeed.net; LARTC
Subject: Re: [LARTC] TC Hashing Filters
Adam Towarnyckyj wrote:>
> Hey all,
> So I got the script to run and populate everything. I watch as
> the script uses the tc commands to add filters for each IP into their
> appropriate tables. After 2045 entries, it starts to give me a "File
> Exists" error. I''ve done extensive testing on the script and
everything
COMMENT: Why is it that everyone who has a bad script does not post
(even a snip of) it?
The limit is 65536.
If you don''t get the significance of that, it means the numbers are hex
and you''re trying to make ''em be decimal. Go read docum.org.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Tue, 13 Jul 2004, Adam Towarnyckyj wrote:> First off, no need to be rude. I read the documentation; I just missed > the number limit. > Second, I know it''s in hex and that''s what I''m using. > Third, the script is rather large. I didn''t want to post even a snippet > because it is so big. But, if that''s what you require in order to give > me some advice, then here it is:Show me the command that fails.> > Action("$tc qdisc del dev $dev root"); > Action("$tc qdisc add dev $dev root handle 1:0 cbq bandwidth 200mbit > avpkt 1000"); > > # Create ''transit class'', tc hash tables, and hash filter > > Action("$tc class add dev $dev parent 1: classid 1:2 cbq bandwidth > 200Mbit rate 200MBit allot 1514 weight 2Mbit prio 8 maxburst 20 avpkt > 1000"); > Action("$tc filter add dev $dev parent 1: handle 2: protocol ip u32 > divisor 256"); > Action("$tc filter add dev $dev protocol ip parent 1: u32 match ip dst > 0.0.0.0/0 hashkey mask 0x000000ff at 16 link 2:"); > > # Create classes for rate groups > > @RATES = SelectSQL("SELECT dsrate FROM dsrate"); > my $classid = 3; > foreach $dsrate (@RATES) { > $ds = $$dsrate{dsrate}; > if ($ds == "0" || $ds == "1") { > next; > } > Action("$tc class add dev eth1 parent 1: classid 1:$classid cbq > bandwidth 200Mbit rate $$dsrate{dsrate}Kbit allot 1514 prio 5 maxburst > 20 avpkt 1000 bounded"); > $rates{$ds} = $classid; > $classid++; > } > > # Get our list of accounts > > @MODEMS = SelectSQL("SELECT mid, dsrate FROM modems"); > > $z = 0; > # Figure out account IPs and put ''em in! > foreach $modem (@MODEMS) { > if ($$modem{dsrate} == "0" || $$modem{dsrate} == "1") { > next; > } > my @COMPUTERS = SelectSQL("SELECT ipid FROM computers WHERE > mid=''$$modem{mid}''"); > foreach $computer (@COMPUTERS) { > my $ip = SelectSingleSQL("SELECT ipaddr FROM ips WHERE > ipid=''$$computer{ipid}''"); > @octets = split(/\./,$ip); > $table = $octets[3]; > $table = sprintf("%X", $table); > $classid = $rates{$$modem{dsrate}}; > Action("$tc filter add dev $dev protocol ip parent 1: > u32 ht 2:$table: match ip dst $ip flowid 1:$classid"); > $z++; > print "$z\n"; > } > } > > "Action" is a sub that performs a system action and reports errors to > another sub. > "SelectSQL" is a sub that performs a sql query to our database. > > COMMENT: Why is it whenever anyone new posts a question to a list even > after reading through the documentation, someone always has to jump down > his throat instead of being helpful and kind? This is a mailing list > which was created so people can ask questions and get a helpful response > from a community. I have been nothing but nice to everyone here and I am > really grateful for all the help that has been provided so far. Instead > of barking at people, simply stating that you require a snippet of code > and providing an answer would be much more helpful and wouldn''t cause > people like me to bark back which is surely going to cause an argument > between us. > > Thanks again! > Adam Towarnyckyj > > > -----Original Message----- > From: gypsy [mailto:gypsy@iswest.com] > Sent: Monday, July 12, 2004 6:24 PM > To: adamt@commspeed.net; LARTC > Subject: Re: [LARTC] TC Hashing Filters > > Adam Towarnyckyj wrote: >> >> Hey all, >> So I got the script to run and populate everything. I watch as >> the script uses the tc commands to add filters for each IP into their >> appropriate tables. After 2045 entries, it starts to give me a "File >> Exists" error. I''ve done extensive testing on the script and > everything > > COMMENT: Why is it that everyone who has a bad script does not post > (even a snip of) it? > > The limit is 65536. > > If you don''t get the significance of that, it means the numbers are hex > and you''re trying to make ''em be decimal. Go read docum.org. > > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >--- Catalin(ux aka Dino) BOIE catab at deuroconsult.ro http://kernel.umbrella.ro/ _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Adam Towarnyckyj wrote:> > First off, no need to be rude. I read the documentation; I just missed > the number limit.I apolgize. I regretted hitting SEND as soon as I came to my senses and I''m ashamed of how long it took me to do that. I had a (what turned out to be minor) emergency last night, I overslept this morning and I have a prior committment tonight so I can''t get to this until at least tomorrow night. But I do have a couple of general suggestions now:> Second, I know it''s in hex and that''s what I''m using.Try setting classid to 0xffff and decrement rather than increment it. Pepper the script with debug (print or echo) lines error traps that exit the script on error. Is what you posted entire so that if I extract it from the message then it should run?> Action("$tc qdisc del dev $dev root"); > Action("$tc qdisc add dev $dev root handle 1:0 cbq bandwidth 200mbit > avpkt 1000");gypsy _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
gypsy wrote:>Try setting classid to 0xffff and decrement rather than increment it. > >Pepper the script with >debug (print or echo) lines >error traps that exit the script on error. > >Is what you posted entire so that if I extract it from the message then >it should run?It''s really tough for me to convey what I''ve done so far mainly because when I get in "Troubleshoot Mode" I really tend to forget what I''ve tried and the results of those efforts after I''ve done them. This is because if it doesn''t work, I rule it out and go on to the next thing until I find out what the problem is. The reason I say this is because I didn''t really tell you specifically what I''ve done to troubleshoot. Only that I found out where the problem was. For this, I apologize. So let me try to be as specific as possible. This is what the script does step by step: 1. Connect to the provisioning database (MySQL) 2. Define subroutines a. SelectSQL - Subroutine for placing all information from a sql query into a variable. b. SelectSingleSQL - Subroutine for placing one piece of information from a sql query into a variable. c. SimpleSQL - Subroutine for making a sql query d. Action - Subroutine for performing a system action and outputting any errors to an array for later use. 3. Remove existing root qdisc and add a new one (clears all information currently stored). 4. Create transit class and hash table/filter. 5. For creating the individual classes for each rate, we have it connect to our database and add a class for each rate located in that database. This is so it can be dynamic in case we need to add new classes down the road. 6. This is where the script grabs all the accounts from our database by modem. The modem table holds the rate for each customer. Then the script compares those modems to the public IPs assigned to that customer and adds the tc command to limit that IP based on the modem rate. 7. Take all the errors from any "Action" and output them. (This emails to me directly when there''s a problem). 8. There''s a bunch of stuff here for promotional rates we''re running that is unimportant to the current problem I''m having. The error appears at step 6. For each modem in our database, it checks the IPs assigned to it. For each of those, it runs: tc filter add dev $dev protocol ip parent 1: u32 ht 2:$table: match ip dst $ip flowid 1:$classid I added a counter in there and an exit command in the "Action" subroutine so that when the script errors, it exits and shows me how many IPs tc has added before it produced an error. The number was 2045 and the error was RTNETLINK answers: File exists. As for your suggestion about the classid, I''m a bit confused as to what you mean about decrementing it. Could you be a little more specific on where this is in the script? I have attached the script in its entirety so you can see it and maybe figure out what is wrong. The only problem with running it would be connecting to the database. If you want, I can put up a mock database and you can connect to that for testing purposes. The problem with this is that I''d have to populate it with about 3000 entries for you to see the error I''m seeing. I have removed my traps and counters so you can see what the script was originally. I''ll comment where the error occurs. Thank you all once again for your help and time. It is very much appreciated. #!/usr/bin/perl # # TC Helper Script: Written by Mike Davis & Adam Towarnyckyj # # Synchronizes data rates with MySQL server and applies hourly. # ### Configuration Section ### $dev = "eth1"; $tc = "/sbin/tc"; $mysql_host = "sql database"; $mysql_db = "databse"; $mysql_user = "user"; $mysql_pass = "password"; ### END Configuration Section ### use POSIX qw(strftime); # Database connect and define subroutines use DBI; $dsn = "DBI:mysql:database=$mysql_db;hostname=$mysql_host"; $dbh = DBI->connect($dsn, $mysql_user, $mysql_pass) || die "Can''t connect to database: " . DBI->errstr; # Subroutine for placing all information from a sql query into a variable. sub SelectSQL { my($sql) = @_; my @MATCHES, $hash; $sth = $dbh->prepare("$sql"); $sth->execute(); while ($hash = $sth->fetchrow_hashref) { push @MATCHES, $hash; } return @MATCHES; } sub SelectSingleSQL { my($sql) = @_; my($gotit, $return, $hash); $sth = $dbh->prepare("$sql"); $sth->execute(); while ($hash = $sth->fetchrow_array) { unless ($gotit) { $return = $hash; $gotit++; } else { warn "got multiple SQL returns when exepecting only one"; } } return $return; } sub SimpleSQL { my($sql) = $_[0]; my $rows_affected; $rows_affected = $dbh->do($sql); return $rows_affected; } sub Action { my($action) = @_; # print"Performing: $action\n"; $warn=`$action 2>&1`; if ($warn) { chomp($warn); $prepare = "ERROR: $warn. Command was: $action"; # print"WHOOPS: $warn\n"; push @WARNING, $prepare; } } ### Ok, now we start having fun. Let''s rebuild the tc tree. # Remove existing tree and add the root. Action("$tc qdisc del dev $dev root"); Action("$tc qdisc add dev $dev root handle 1:0 cbq bandwidth 200mbit avpkt 1000"); # Create ''transit class'', tc hash tables, and hash filter Action("$tc class add dev $dev parent 1: classid 1:2 cbq bandwidth 200Mbit rate 200MBit allot 1514 weight 2Mbit prio 8 maxburst 20 avpkt 1000"); Action("$tc filter add dev $dev parent 1: handle 2: protocol ip u32 divisor 256"); Action("$tc filter add dev $dev protocol ip parent 1: u32 match ip dst 0.0.0.0/0 hashkey mask 0x000000ff at 16 link 2:"); # Create classes for rate groups @RATES = SelectSQL("SELECT dsrate FROM dsrate"); my $classid = 3; foreach $dsrate (@RATES) { $ds = $$dsrate{dsrate}; if ($ds == "0" || $ds == "1") { next; } Action("$tc class add dev eth1 parent 1: classid 1:$classid cbq bandwidth 200Mbit rate $$dsrate{dsrate}Kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded"); $rates{$ds} = $classid; $classid++; } # Get our list of accounts @MODEMS = SelectSQL("SELECT mid, dsrate FROM modems"); # Figure out account IPs and put ''em in! foreach $modem (@MODEMS) { if ($$modem{dsrate} == "0" || $$modem{dsrate} == "1") { next; } my @COMPUTERS = SelectSQL("SELECT ipid FROM computers WHERE mid=''$$modem{mid}''"); foreach $computer (@COMPUTERS) { my $ip = SelectSingleSQL("SELECT ipaddr FROM ips WHERE ipid=''$$computer{ipid}''"); @octets = split(/\./,$ip); $table = $octets[3]; $table = sprintf("%X", $table); $classid = $rates{$$modem{dsrate}}; Action("$tc filter add dev $dev protocol ip parent 1: u32 ht 2:$table: match ip dst $ip flowid 1:$classid"); ### Here is where it errors after 2045 entries ### } } if (@WARNING) { print"WARNING: TCHELPER produced errors! See below:\n @WARNING\n"; } # # Cool, everyone is now limited. Let''s do some up-keep on the promo rates. # # First we check accounts with a promo rate and no promo code, and fill it in. @PROMORATES = SelectSQL("SELECT dsrate,drpromo FROM dsrate WHERE drpromo !=''0''"); $month=strftime "%m", localtime; $day=strftime "%d", localtime; $year=strftime "%Y", localtime; $today="$year" . "$month" . "$day"; foreach $rate (@PROMORATES) { ($exprate,$expdays) = split("-",$$rate{drpromo}); $expdate = $day + $expdays; $expmonth = $month; $expyear = $year; while ($expdate > "30") { $expmonth++; $expdate = $expdate - 30; } while ($expmonth > "12") { $expyear++; $expmonth = $expmonth - 12; } $absexpdate = "$expyear" . "$expmonth" . "$expdate"; @UNMARKED = SelectSQL("SELECT mid FROM modems WHERE dsrate=''$$rate{dsrate}'' AND promocode=''''"); foreach $mark (@UNMARKED) { $query = SimpleSQL("UPDATE modems SET promocode=''$exprate-$absexpdate'' WHERE mid=''$$mark{mid}''"); } } # Now we check for expired promo codes and reset their rate. @PROMOACCTS = SelectSQL("SELECT mid,promocode FROM modems WHERE promocode != ''''"); foreach $acct (@PROMOACCTS) { ($exprate,$expdate) = split("-",$$acct{promocode}); if ($expdate <= $today) { $query = SimpleSQL("UPDATE modems SET dsrate=''$exprate'', promocode='''' WHERE mid=''$$acct{mid}''"); } } # Exit Nice and clean. $dbh->disconnect; exit(0); _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Wednesday 14 July 2004 14:06, Adam Towarnyckyj wrote: <snip>> As for your suggestion about the classid, I''m a bit confused as > to what you mean about decrementing it. Could you be a little more > specific on where this is in the script? ><snip script> I think he means start with classid 0xffff and then substract one as you iterate through each row in the table. So next you''d use 0xfffe and so on. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Adam Towarnyckyj wrote:> > gypsy wrote: > >Try setting classid to 0xffff and decrement rather than increment it. > > > >Pepper the script with > >debug (print or echo) lines > >error traps that exit the script on error. > > The error appears at step 6. For each modem in our database, it > checks the IPs assigned to it. For each of those, it runs: tc filter add > dev $dev protocol ip parent 1: u32 ht 2:$table: match ip dst $ip flowid > 1:$classidI think that "$classid" is outside its allowable range when that line executes, although it is also possible that the RTNETLINK message is due to an erroneous value in "$dev", "$ip" or "$table". Your script can be modified to display the line with the substitutions and to exit when executing causes an error. What I envision is that you''ll watch 2000+ lines of good stuff scroll by and that the final line will be your Bad Boy. And the last few lines are what we all are holding our breath to see <grin>.> I added a counter in there and an exit command in the "Action" > subroutine so that when the script errors, it exits and shows me how > many IPs tc has added before it produced an error. The number was 2045 > and the error was RTNETLINK answers: File exists.I still don''t think it is how many. I am certain that some value is just not in the expected range.> As for your suggestion about the classid, I''m a bit confused as > to what you mean about decrementing it. Could you be a little more > specific on where this is in the script? > Action("$tc class add dev eth1 parent 1: classid 1:$classid cbq > bandwidth 200Mbit rate $$dsrate{dsrate}Kbit allot 1514 prio 5 maxburst > 20 avpkt 1000 bounded"); > $rates{$ds} = $classid; > $classid++;I think the line above should count down from 65535, not up.> ipid=''$$computer{ipid}''"); > @octets = split(/\./,$ip); > $table = $octets[3]; > $table = sprintf("%X", $table); > $classid = $rates{$$modem{dsrate}}; > Action("$tc filter add dev $dev protocol ip parent 1: > u32 ht 2:$table: match ip dst $ip flowid 1:$classid"); ### Here is > where it errors after 2045 entries ###And here is where you need to display the Action line. Plus add an error trap to exit when it fails. Sorry, that''s all I have time for now. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Wed, Jul 14, 2004 at 07:13:11PM -0700, gypsy wrote:> Your script can be modified to display the line with the substitutions > and to exit when executing causes an error. What I envision is that > you''ll watch 2000+ lines of good stuff scroll by and that the final line > will be your Bad Boy. And the last few lines are what we all are > holding our breath to see <grin>.This took me a little while to figure out myself, so I thought I''d post it. If you are running a bash script, what you want to do is to add at the top: set -e # causes script exit on error set -x # causes command to be printed before it is executed Glen -- ****************************************************************** Glen W. Mabey Glen.Mabey@usu.edu http://mabeys.homelinux.com/glen/ ****************************************************************** _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
gypsy wrote:> Your script can be modified to display the line with the substitutions > and to exit when executing causes an error. What I envision is that > you''ll watch 2000+ lines of good stuff scroll by and that the finalline> will be your Bad Boy. And the last few lines are what we all are > holding our breath to see <grin>.I have done as you suggested (actually, if you noticed in the Action subroutine, all the error reporting from before was still there I just commented it out) and this is what I''ve come up with. This is the last known good command and then the command that errors, as well as the error message, and the counter I added back in. Performing: /sbin/tc filter add dev eth1 protocol ip parent 1: u32 ht 2:54: match ip dst 216.19.20.84 flowid 1:FFFB 2045 Performing: /sbin/tc filter add dev eth1 protocol ip parent 1: u32 ht 2:2E: match ip dst 216.19.46.46 flowid 1:FFFB WHOOPS: RTNETLINK answers: File exists The error exists on the 2046th command. Any tc filter command I try to add from here on out gives me the same error even if I try it manually instead of using the script. I examined the previous commands that worked fine and there are plenty that dump into the 2:2E: table.> I think the line above should count down from 65535, not up.As you can see, I did this as you suggested. I like it better because it keeps things very separate so there''s no confusion. Thanks for the suggestion.> Sorry, that''s all I have time for now.Any time you can spare is incredibly helpful. Thank you. Adam Towarnyckyj _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Adam Towarnyckyj wrote:> Any tc filter command I try to > add from here on out gives me the same error even if I try it manually > instead of using the script.Adam, That is a killer. Please read the following and then, when we both have the same information, I''m going to try again to assist you. HINT TO READERS: I hope someone else will help us both because I have obligations that I''m stealing time from that I can ill afford. There is a LARTC mailing list thread dated (about) 24 thru 26 June 2003 between Trevor Warren and Michael Ulitskiy whose Subject is "u32 clarification...limits on 2000>???" Please use your favorite method to find it. Note that there may be kernel issues not mentioned by them; kernels change. I''d like to suggest that you see if anyone involved in that thread will send you a testing script; perhaps you could find a way to start with a working setup and then apply minor changes until either it breaks or it suits you. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
ibro tj
2004-Jul-16 18:18 UTC
Shaping using both public and private Ip addresss on same linux box
I have a qos system that is currently shaping (down/up) using public ip address on eth0 and eth1 -eth0 faces the internet.What I want to do is similar to what jiri fojtasek ( http://hyperfighter.jinak.cz/qos ) in his imq implementation example. But in my case , i do not want to use imq but rather attach the 3rd NIC eth2 that will shape traffic originating/destined to my local LAN -192.168.0.0/24. wHAT i find challenging is how to construct the tc scripts between the interface eth0 and eth2 taking care of up/down as well , without conflicting with the public aspect. A sample script will be of great help. Ibrahim T __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - 100MB free storage! http://promotions.yahoo.com/new_mail _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Adam Towarnyckyj wrote:> > Gypsy, > Looks like the script ran fine and everything''s in there. It > took almost an hour to complete. I still believe it might have something > to do with my classes setup. I''ll see if I can modify the script to do > one class per IP like this one does. Thanks! > > AdamAdam, While the limit (if any) on the number of filters is at least 64,516 it is possible that there is a smaller limit for U32 matches or parents or classIDs or _something_. If you find such to be the case, you should report it to this list and to the kernel mailing list. Your success does mean that your software is good. Some time last year (I think), Red Hat released some broken stuff that caused "File exists" errors. You just removed my fears about that issue. Gypsy _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Dear list, After much code crunching and beating my head against the wall (literally), I discovered the faulty code. Thanks mostly to gypsy who played a huge role in helping me discover the problem. My tc filter line had an error in it which completely baffles me because it worked up until 2045 lines were entered. Old command: tc filter add dev $dev protocol ip parent 12: u32 ht 2:$table1: ht 3:$table2: match ip dst $ip/32 flowid 12:$classid New command: tc filter add dev $dev protocol ip parent 12: prio 5 u32 ht 2:$table1: ht 3:$table2: match ip dst $ip/32 flowid 12:$classid Who would have thought that a prio would cause that much trouble? I mean, I know in the documentation it says that it is required for CLASS commands, but since there''s nothing on filters, I had no clue. And usually if a command requires a parameter, it will error immediately and not wait for 2045 entries. Also, why the hell would the priority cause a problem like that to begin with?! Weird. Anyways, THANKS EVERYONE! (gypsy, Catalin, and Ed especially) It all works, it''s very resource friendly, and all is good again (and I get to keep my job)! The processor is only about 20% in use now. I''m very grateful for the hashing filter suggestion from Ed. Thanks a ton. I will be posting a How To shortly so others can do the same if they ever need to. Thanks again! Adam Towarnyckyj -----Original Message----- From: lartc-admin@mailman.ds9a.nl [mailto:lartc-admin@mailman.ds9a.nl] On Behalf Of gypsy Sent: Thursday, July 15, 2004 8:37 PM To: adamt@commspeed.net; LARTC Subject: Re: [LARTC] TC Hashing Filters Adam Towarnyckyj wrote:> Any tc filter command I try to > add from here on out gives me the same error even if I try it manually > instead of using the script.Adam, That is a killer. Please read the following and then, when we both have the same information, I''m going to try again to assist you. HINT TO READERS: I hope someone else will help us both because I have obligations that I''m stealing time from that I can ill afford. There is a LARTC mailing list thread dated (about) 24 thru 26 June 2003 between Trevor Warren and Michael Ulitskiy whose Subject is "u32 clarification...limits on 2000>???" Please use your favorite method to find it. Note that there may be kernel issues not mentioned by them; kernels change. I''d like to suggest that you see if anyone involved in that thread will send you a testing script; perhaps you could find a way to start with a working setup and then apply minor changes until either it breaks or it suits you. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/