We have a Linux box that is acting as the gateway to the internet for about 400 people, typically there are not more then 50 of them using the internet at any given time. We would like to provide different levels of access to different users. For example 128kbps to some users and 256kbps to others. We have considered creating a class and qdisc for each user (using htb) however we don''t know how much overhead creating 50-200 classes and qdiscs would involve, would this put too much strain on the Linux box? Is it better to create fewer classes and qdisc and assign multiple users to each? I haven''t been able to find any test on maximum effect number of qdiscs, but it could be I have just been looking in the wrong place. If any one has any ideas or could point me in the right direction it would be greatly appreciated. Spencer
Spencer wrote:> > Is it > better to create fewer classes and qdisc and assign multiple users to each? > I haven''t been able to find any test on maximum effect number of qdiscs, but > it could be I have just been looking in the wrong place. If any one has any > ideas or could point me in the right direction it would be greatly > appreciated. > > SpencerYou''re not the first person to ask this. AFAIK there is no benchmark. People just do it. I suggest googling this ML for "hash", "internet cafe", "pyshaper", "PaceMaker" and whatever else that leads to. IIRC "hotel" may also be a good search word. Tomasz Paszkowski runs a HUGE script for his HFSC setup. The short answer is that, if you can create a hash that matches, you can reduce the volume of entries; but that is more a convenience than something necessary for efficiency. It takes a HELL of a lot to make Linux groan under the load. I once spent > 1 hour loading ~32K filters, but when the script finished, I could not tell they were there based on the performance of my AMD Duron 1400 CPU, 256Mb RAM equipped Linux box. The following is probably the most useful single site you''ll find: http://digriz.org.uk/ -- gypsy
Spencer wrote:> We have a Linux box that is acting as the gateway to the internet for about > 400 people, typically there are not more then 50 of them using the internet > at any given time. We would like to provide different levels of access to > different users. For example 128kbps to some users and 256kbps to others. > We have considered creating a class and qdisc for each user (using htb) > however we don''t know how much overhead creating 50-200 classes and > qdiscs would involve, would this put too much strain on the Linux box? Is > it > better to create fewer classes and qdisc and assign multiple users to each? > I haven''t been able to find any test on maximum effect number of qdiscs, but > it could be I have just been looking in the wrong place. If any one has any > ideas or could point me in the right direction it would be greatly > appreciated.I have P4 3.0 GHz, 1 GB RAM. I have 3500 potential users (top load about 800 users, average 400). I have 3 interfaces (2 WAN + 1 LAN), so I have 10500 queues total (3500 on each interface). The traffic is 24Mbit max, average 20Mbit. Without u32 hashing my box run at 60-70% CPU utilization. After applying hashing the box is running with 25% top utilization, average 15%. The two thing you must remember when running a box for many users: * use iptables chains. I prefer chains of 30-40 entries. * use u32 hashing. This will greatly improve CPU utilization. About 500-1000% in my case. Szymon Miotk
Szymon Miotk wrote:> I have P4 3.0 GHz, 1 GB RAM. > I have 3500 potential users (top load about 800 users, average 400). I > have 3 interfaces (2 WAN + 1 LAN), so I have 10500 queues total (3500 on > each interface). > The traffic is 24Mbit max, average 20Mbit.OK... What''s wrong? I can only use 4775 filters :/ This is a reslts of my small script... loops: 674 filters: 4718 classes: 2022 loops: 675 filters: 4725 classes: 2025 loops: 676 filters: 4732 classes: 2028 loops: 677 filters: 4739 classes: 2031 loops: 678 filters: 4746 classes: 2034 loops: 679 filters: 4753 classes: 2037 loops: 680 filters: 4760 classes: 2040 loops: 681 filters: 4767 classes: 2043 loops: 682 filters: 4774 classes: 2046 RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel loops: 683 filters: 4781 classes: 2049 RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel RTNETLINK answers: File exists We have an error talking to the kernel loops: 684 filters: 4788 classes: 2052 What''s wrong? I need more filters :/ I have 2.6.11.11 kernel with new iproute2, u32 match mark support and IMQ (AB)... Everyone can make theoretically 0xffff (65535) classes and qdiscs on one device. And I think this is true, but I can''t add more filters then 4775! :( --- v=1; cnt=0; tc qdisc add dev imq0 root handle 1:0 htb while : [ $v -le 11000 ]; do qu0=`printf "%x\n" $v` qu1=`printf "%x\n" $v` qu2=`printf "%x\n" $((v+1))` qu3=`printf "%x\n" $((v+2))` tc class add dev imq0 parent 1:0 classid 1:$qu1 htb rate 1000Mbit ceil 1000Mbit quantum 1500 tc class add dev imq0 parent 1:$qu1 classid 1:$qu2 htb rate 1kbit ceil 1kbit prio 1 quantum 1500 tc class add dev imq0 parent 1:$qu1 classid 1:$qu3 htb rate 1kbit ceil 1kbit prio 2 quantum 1500 tc qdisc add dev imq0 parent 1:$qu2 sfq tc qdisc add dev imq0 parent 1:$qu3 sfq tc filter add dev imq0 protocol ip parent 1:$qu0 pref 5 u32 match ip dst 192.168.0.5 flowid 1:$qu1 tc filter add dev imq0 protocol ip parent 1:$qu1 pref 6 u32 match ip protocol 6 0xff match ip sport 80 0xffff flowid 1:$qu2 tc filter add dev imq0 protocol ip parent 1:$qu1 pref 6 u32 match ip protocol 6 0xff match ip dport 80 0xffff flowid 1:$qu2 tc filter add dev imq0 protocol ip parent 1:$qu1 pref 6 u32 match ip protocol 17 0xff match ip sport 53 0xffff flowid 1:$qu2 tc filter add dev imq0 protocol ip parent 1:$qu1 pref 6 u32 match ip protocol 17 0xff match ip dport 53 0xffff flowid 1:$qu2 tc filter add dev imq0 protocol ip parent 1:$qu1 pref 6 u32 match ip protocol 1 0xff flowid 1:$qu2 tc filter add dev imq0 protocol ip parent 1:$qu1 pref 7 u32 match ip dst 192.168.0.5 flowid 1:$qu3 let "v=v+3" let "cnt=cnt+1" echo "loops: $cnt filters: $((cnt*7)) classes: $((cnt*3))" done
Paweł Staszewski
2005-Jun-03 10:52 UTC
Re: How many (htb) tc classes and qdiscs are too many?
Hello I have 4000 users and i use hfsc for shaping them. Each class has own qdisc(esfq) tc -s -d qdisc show dev vlan0891 | grep qdisc | wc -l 4355 tc -s -d qdisc show dev eth2 | grep qdisc | wc -l 4355 I use hashing filters. System is: P4 3.2GHz (HT enabled) 2GB RAM 2xIntel gigabit (Napi enabled) Machine load is: 12:57:06 up 11:24, 2 users, load average: 0.00, 0.05, 0.06 mpstat -P ALL 1 (output) Linux 2.6.12-rc5-git6 (natjawman) 06/03/05 12:57:24 CPU %user %nice %system %iowait %irq %soft %idle intr/s 12:57:25 all 12.00 0.00 30.50 0.00 0.50 14.50 42.50 4990.00 12:57:25 0 12.00 0.00 32.00 0.00 1.00 13.00 42.00 3390.00 12:57:25 1 12.00 0.00 29.00 0.00 0.00 16.00 42.00 1603.00 12:57:25 CPU %user %nice %system %iowait %irq %soft %idle intr/s 12:57:26 all 11.50 0.00 30.50 0.00 0.50 16.50 41.00 4970.00 12:57:26 0 12.00 0.00 29.00 0.00 0.00 17.00 42.00 3302.00 12:57:26 1 11.00 0.00 33.00 0.00 1.00 16.00 41.00 1666.00 12:57:26 CPU %user %nice %system %iowait %irq %soft %idle intr/s 12:57:27 all 12.94 0.00 29.85 0.00 0.50 14.43 42.29 4998.02 12:57:27 0 12.87 0.00 30.69 0.00 0.99 14.85 40.59 3324.75 12:57:27 1 13.86 0.00 28.71 0.00 0.00 13.86 42.57 1674.26 12:57:27 CPU %user %nice %system %iowait %irq %soft %idle intr/s 12:57:28 all 11.50 0.00 29.00 0.00 0.50 19.00 40.00 4912.87 12:57:28 0 11.88 0.00 31.68 0.00 0.99 15.84 39.60 3304.95 12:57:28 1 10.89 0.00 25.74 0.00 0.00 21.78 40.59 1608.91 Peak bw is 32Mbit/s Average bw 25Mbit/s Machine is doing also SNAT to all clients: iptables -L -n -v -t nat | grep SNAT | wc -l 4465 Some example script which i use for hashing filters is in attachement. Best Regards Paweł Staszewski ART-COM +48327522333 +480609183038>>>lartc-request@mailman.ds9a.nl 06/03/05 8:37 am >>>Send LARTC mailing list submissions to lartc@mailman.ds9a.nl To subscribe or unsubscribe via the World Wide Web, visit http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc or, via email, send a message with subject or body ''help'' to lartc-request@mailman.ds9a.nl You can reach the person managing the list at lartc-owner@mailman.ds9a.nl When replying, please edit your Subject line so it is more specific than "Re: Contents of LARTC digest..." Today''s Topics: 1. Re: how to configure linux in production line (/dev/rob0) 2. Re: HTB on loopback gives a bit rate multiplied by 8 (Kiruthika Selvamani) 3. Re: how to configure linux in production line (Taylor, Grant) 4. iproute + xml (Alberto Torres) 5. Re: HTB on loopback gives a bit rate multiplied by 8 (Andy Furniss) 6. How many (htb) tc classes and qdiscs are too many? (Spencer) 7. Re: [PATCH] Support module autoloading in iproute2 (Stephen Hemminger) 8. Re: How many (htb) tc classes and qdiscs are too many? (threaded) 9. Re: iproute + xml (cristian_dimache@rtanet.ro) 10. Re: How many (htb) tc classes and qdiscs are too many? (Szymon Miotk) ---------------------------------------------------------------------- Message: 1 Date: Thu, 02 Jun 2005 06:34:14 -0500 From: /dev/rob0 <rob0@gmx.co.uk> Subject: Re: [LARTC] how to configure linux in production line To: LARTC@mailman.ds9a.nl Message-ID: <429EEEB6.8050201@gmx.co.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Gonn Star wrote:>I am new in linux world,basically I''m using red hat 9 >kernel 2.4.20-8. I need to build a trusted gateway. myWhoa! You are starting out with something very old and bug-ridden. You should scrap that and switch to a current release, whatever distro you may choose. Quite a few of those old bugs can bite very hard, including root compromises. Being new, did you know how to update for security? Sure, there''s Fedora Legacy which may or may not be supporting the old stuff with updates, but that is intended for people who have long-running stable servers ... not to entice new users to RH 9.>linux box will be the gateway for several machine PCs >to go to the desired server. there will be several >subnets under the linux box, I''ve already assigned >static IPs for the PCs . Now my problem is I only need >2 PCs from each subnets to connect to certain servers, >and those 2 PCs can only have transaction(open) to the >specified servers, for others it will >drop(firewalled). for other PCs, they can''t log on to >the outside world. should I use only iptable rules or >with the help of squid(ACL) as well ?You do not seem to understand that HTTP is just one of many TCP/IP protocols, and yet you want to set up complex networking controls. Anyone who knows more than you do would likely find it a trivial task to get around your controls.>please add up the commands as well. Thanks.Specific questions which show that you have tried will tend to be better-received than generalised requests for spoonfeeding. I do things like this for a living, and I do not have time to earn your living as well. You mention "production" which implies that this is needed in a business setting. If so it''s probably worth it to the business owners to pay for expertise. You can''t learn everything you need to know, overnight. For you, I would recommend starting with the basics. There are good HOWTOs at netfilter.org which might help. -- mail to this address is discarded unless "/dev/rob0" or "not-spam" is in Subject: header ------------------------------ Message: 2 Date: Thu, 2 Jun 2005 09:40:46 -0400 From: Kiruthika Selvamani <kiruthika.selvamani@gmail.com> Subject: Re: [LARTC] HTB on loopback gives a bit rate multiplied by 8 To: Andy Furniss <andy.furniss@dsl.pipex.com> Cc: lartc@mailman.ds9a.nl Message-ID: <5ee6fe6105060206401e1f1ff@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Hi Andy, Thanks for the suggestion. I changed the MTU to 1500 and it started working. Is this because HTB shapes traffic based on packet rate rather than bit rate? How does it use the rate lookup tables? Thanks Kiruthika On 6/1/05, Andy Furniss <andy.furniss@dsl.pipex.com> wrote:>Kiruthika Selvamani wrote: >>Hi, >> >>I am trying to use htb to limit bandwidth on loopback for traffic >>through particular port. >> >>Here is the script I am using. >> >>tc qdisc add dev lo root handle 1: htb >>tc class add dev lo parent 1: classid 1:1 htb rate 100kbit ceil100kbit>>tc class add dev lo parent 1:1 classid 1:10 htb rate 50kbit ceil50kbit>>tc class add dev lo parent 1:1 classid 1:11 htb rate 50kbit ceil50kbit>>tc filter add dev lo protocol ip parent 1:0 prio 0 u32 match ip sport >>22 0xffff flowid 1:10 >>tc filter add dev lo protocol ip parent 1:0 prio 0 u32 match ip dport >>22 0xffff flowid 1:11 >> >>When this script is applied across eth0 (when I do a sftp to another >>machine) the bandwidth limitation is applied correctly. However if I >>use this in loopback (sftp to another directory in the same machine) >>then I get bit rate approx 400kbit - i.e. usually it roughly >>multiplies the bit rate by 8. Why does this happen? Does HTB work >>differently in loopback? Any clue regarding this would be mostl >>helpful. > >It''s because the MTU on lo is big and htb uses a small one when it asks>tc to make it''s rate lookup tables. > >if you do a tc -s class ls dev lo you will see there is a giantscounter>, giant packets are only limited as if they are the size of the biggest>slot in the lookup table. > >To fix specify the mtu of lo on the htb classes or set the mtu on lo to>1500. > >Andy. >------------------------------ Message: 3 Date: Thu, 02 Jun 2005 10:46:48 -0500 From: "Taylor, Grant" <gtaylor@riverviewtech.net> Subject: Re: [LARTC] how to configure linux in production line To: LARTC@mailman.ds9a.nl Message-ID: <429F29E8.4070805@riverviewtech.net> Content-Type: text/plain; charset=ISO-8859-1 Gonn Star wrote:>I am new in linux world,basically I''m using red hat 9 >kernel 2.4.20-8. I need to build a trusted gateway. my >linux box will be the gateway for several machine PCs >to go to the desired server. there will be several >subnets under the linux box, I''ve already assigned >static IPs for the PCs . Now my problem is I only need >2 PCs from each subnets to connect to certain servers, >and those 2 PCs can only have transaction(open) to the >specified servers, for others it will >drop(firewalled). for other PCs, they can''t log on to >the outside world. should I use only iptable rules or >with the help of squid(ACL) as well ? please add up >the commands as well. Thanks.This sounds like a fairly basic firewall with out Squid in the mix. In short you are probably looking at a firewall like this (NOTE: This script will be incomplete for just about any scenario, but will give you the idea.): iptables -t filter -P FORWARD DROP iptables -t filter -F FORWARD iptables -t filter -A FORWARD -s 192.168.0.1 -j ACCEPT iptables -t filter -A FORWARD -s 192.168.0.2 -j ACCEPT iptables -t filter -A FORWARD -s 192.168.1.1 -j ACCEPT iptables -t filter -A FORWARD -s 192.168.1.2 -j ACCEPT iptables -t filter -A FORWARD -s 192.168.2.1 -j ACCEPT iptables -t filter -A FORWARD -s 192.168.2.2 -j ACCEPT iptables -t filter -A FORWARD -j REJECT --reject-with icmp-net-unreachable This quick and dirty (and incomplete) script will set the default policy (-P) of the FORWARD chain to DROP all traffic that is to be forwarded and not handled by any other rule. Once the default policy has been set it flushes (-F) the FORWARD chain to make sure that there were not any old rules lingering arround that could mess things up. The next six rules are in place to explicietly allow just the two machines from three subnets (in this example) to pass traffic through the FORWARD chain on out to a different network. Any traffic that is not explicietly handled by the six rules to allow traffic to be forwarded will meat the last rule which will reject the traffic with a message saying that there is no route to the destination thus making the computers think that they are icolated. As someone else pointed out if you are new to the Linux community you might be better off served by finding someone in your area with more experience at hardening a box and a firewall to help you in this endevor. Or if you are not new to unix or firewalling, just Linux and you need to acclimate your self with the Linux syntax and methodology you will probably be ok. Either way it would probably be worth your time to skim some of the HOW-TOs that are out there, namely the NetFilter HOW-TO as you are asking questions that are answered in it. Grant. . . . ------------------------------ Message: 4 Date: Thu, 2 Jun 2005 21:22:19 +0200 From: Alberto Torres <perezoso@gmail.com> Subject: [LARTC] iproute + xml To: lartc@mailman.ds9a.nl Message-ID: <850c9dea05060212225a0d8549@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Hello there, i am continuing with the development of the iproute GUI. I was wondering if there is a xml parser for the set up of the queues. I have been searching but i cant find any... anyone? ------------------------------ Message: 5 Date: Thu, 02 Jun 2005 20:32:25 +0100 From: Andy Furniss <andy.furniss@dsl.pipex.com> Subject: Re: [LARTC] HTB on loopback gives a bit rate multiplied by 8 To: Kiruthika Selvamani <kiruthika.selvamani@gmail.com> Cc: lartc@mailman.ds9a.nl Message-ID: <429F5EC9.5020202@dsl.pipex.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Kiruthika Selvamani wrote:>Hi Andy, >Thanks for the suggestion. I changed the MTU to 1500 and it started >working. Is this because HTB shapes traffic based on packet rate >rather than bit rate? How does it use the rate lookup tables?It''s not based on packet rate as such, the lookup tables are for the time delay for different packet lengths at the different rates. There is one for each rate and ceil pre calculated for efficiency. Each table has 256 slots so the mtu is needed to fill it efficiently, with normal mtu each slot is 8 bytes apart. If you had told htb the mtu of lo (16436) then each slot would have been calculated to cover a bigger range of bytes. I suppose the giants counter is a warning that these packets are not being shaped properly as they are too big. I suppose devik decided to do this in preference to calculating the delay for every giant so it didn''t slow things down too much. Personally I am glad he didn''t just use the interface mtu, as my dsl ppp0 gets one of 32k - it never sees a packet bigger than 1500 though, so if htb used 32k the shaping of small packets would be too innacurate. Andy. ------------------------------ Message: 6 Date: Thu, 2 Jun 2005 16:07:31 -0600 From: "Spencer" <spencer@hotsitenet.com> Subject: [LARTC] How many (htb) tc classes and qdiscs are too many? To: <lartc@mailman.ds9a.nl> Message-ID: <000501c567bf$792fd700$650fa8c0@hotsitespencer> Content-Type: text/plain;charset="iso-8859-1" We have a Linux box that is acting as the gateway to the internet for about 400 people, typically there are not more then 50 of them using the internet at any given time. We would like to provide different levels of access to different users. For example 128kbps to some users and 256kbps to others. We have considered creating a class and qdisc for each user (using htb) however we don''t know how much overhead creating 50-200 classes and qdiscs would involve, would this put too much strain on the Linux box? Is it better to create fewer classes and qdisc and assign multiple users to each? I haven''t been able to find any test on maximum effect number of qdiscs, but it could be I have just been looking in the wrong place. If any one has any ideas or could point me in the right direction it would be greatly appreciated. Spencer ------------------------------ Message: 7 Date: Thu, 02 Jun 2005 17:20:44 -0700 From: Stephen Hemminger <shemminger@osdl.org> Subject: [LARTC] Re: [PATCH] Support module autoloading in iproute2 To: jt@hpl.hp.com Cc: lartc@mailman.ds9a.nl Message-ID: <429FA25C.2030804@osdl.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Use module aliases and the kernel will do the autoloading. Most distros add something like: alias eth0 e100 to /etc/modprobe.conf ------------------------------ Message: 8 Date: Thu, 02 Jun 2005 19:55:37 -0700 From: threaded <gypsy@iswest.com> Subject: Re: [LARTC] How many (htb) tc classes and qdiscs are too many? To: lartc@mailman.ds9a.nl Cc: Spencer <spencer@hotsitenet.com> Message-ID: <429FC6A9.9F1A5BFE@iswest.com> Content-Type: text/plain; charset=us-ascii Spencer wrote:> >Is it >better to create fewer classes and qdisc and assign multiple users toeach?>I haven''t been able to find any test on maximum effect number ofqdiscs, but>it could be I have just been looking in the wrong place. If any onehas any>ideas or could point me in the right direction it would be greatly >appreciated. > >SpencerYou''re not the first person to ask this. AFAIK there is no benchmark. People just do it. I suggest googling this ML for "hash", "internet cafe", "pyshaper", "PaceMaker" and whatever else that leads to. IIRC "hotel" may also be a good search word. Tomasz Paszkowski runs a HUGE script for his HFSC setup. The short answer is that, if you can create a hash that matches, you can reduce the volume of entries; but that is more a convenience than something necessary for efficiency. It takes a HELL of a lot to make Linux groan under the load. I once spent > 1 hour loading ~32K filters, but when the script finished, I could not tell they were there based on the performance of my AMD Duron 1400 CPU, 256Mb RAM equipped Linux box. The following is probably the most useful single site you''ll find: http://digriz.org.uk/ -- gypsy ------------------------------ Message: 9 Date: Fri, 3 Jun 2005 08:34:31 +0300 (EEST) From: cristian_dimache@rtanet.ro Subject: Re: [LARTC] iproute + xml To: LARTC@mailman.ds9a.nl Message-ID: <50532.194.102.203.14.1117776871.squirrel@194.102.203.14> Content-Type: text/plain;charset=iso-8859-1 Let us look back on the archives: On 12 Jul 2001 17:41:42 -0500, Nikolai Vladychevski wrote:>But what I am trying to do is to release it for >production where the end users would point & click for filter creation&>bandwidth definition, so I think it will be an adventure, but I am >accepting the risks... after all.... it''s free code....I''ve been working on an XML format for describing a traffic control configuration in-house. We''re working on a good way to describe the ru --=__PartD5F63A69.0__Content-Type: application/x-sh; name="outbw.sh" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="outbw.sh" IyEvYmluL3NoCgp0YyBxZGlzYyBkZWwgZGV2IHZsYW4wODkxIHJvb3QKCnRjIHFkaXNjIGFkZCBk ZXYgdmxhbjA4OTEgcm9vdCBoYW5kbGUgMTE6MCBoZnNjIGRlZmF1bHQgODE5MgoKdGMgY2xhc3Mg YWRkIGRldiB2bGFuMDg5MSBwYXJlbnQgMTE6MCBjbGFzc2lkIDExOjEgaGZzYyBscyBtMiAyNTYw a2JwcyB1bCBtMiAyNTYwa2JwcwoKdGMgY2xhc3MgYWRkIGRldiB2bGFuMDg5MSBwYXJlbnQgMTE6 MSBjbGFzc2lkIDExOjMgaGZzYyBscyBtMiAyNTYwa2JwcyB1bCBtMiAyNTYwa2JwcyAKCnRjIGNs YXNzIGFkZCBkZXYgdmxhbjA4OTEgcGFyZW50IDExOjEgY2xhc3NpZCAxMTo4MTkyIGhmc2MgbHMg bTIgMWtiaXQgdWwgbTIgMWtiaXQKCnRjIHFkaXNjIGFkZCBkZXYgdmxhbjA4OTEgcGFyZW50IDEx OjgxOTIgc2ZxIHF1YW50dW0gMTUwMCBwZXJ0dXJiIDEyMAp0YyBxZGlzYyBhZGQgZGV2IHZsYW4w ODkxIHBhcmVudCAxMTozIHNmcSBxdWFudHVtIDE1MDAgcGVydHVyYiAxMjAKIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKCgoKdGMgZmlsdGVyIGFkZCBkZXYgdmxh bjA4OTEgcGFyZW50IDExOjAgcHJpbyAxIHByb3RvY29sIGlwIHUzMgp0YyBmaWx0ZXIgYWRkIGRl diB2bGFuMDg5MSBwYXJlbnQgMTE6MCBwcmlvIDEgaGFuZGxlIDE6IHByb3RvY29sIGlwIHUzMiBk aXZpc29yIDI1NgoKdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50 IDExOjAgcHJpbyAyIHUzMiBodCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4eC40OC4wLzI0IGhh c2hrZXkgbWFzayAweDAwMDAwMGZmIGF0IDEyIGxpbmsgMToKdGMgZmlsdGVyIGFkZCBkZXYgdmxh bjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50IDExOjAgcHJpbyAyIHUzMiBodCA4MDA6OiBtYXRjaCBp cCBzcmMgeHh4Lnh4eC40OS4wLzI0IGhhc2hrZXkgbWFzayAweDAwMDAwMGZmIGF0IDEyIGxpbmsg MToKdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50IDExOjAgcHJp byAyIHUzMiBodCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4eC41MC4wLzI0IGhhc2hrZXkgbWFz ayAweDAwMDAwMGZmIGF0IDEyIGxpbmsgMToKdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJv dG9jb2wgaXAgcGFyZW50IDExOjAgcHJpbyAyIHUzMiBodCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4 Lnh4eC41MS4wLzI0IGhhc2hrZXkgbWFzayAweDAwMDAwMGZmIGF0IDEyIGxpbmsgMToKdGMgZmls dGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50IDExOjAgcHJpbyAyIHUzMiBo dCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4eC41Mi4wLzI0IGhhc2hrZXkgbWFzayAweDAwMDAw MGZmIGF0IDEyIGxpbmsgMToKdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAg cGFyZW50IDExOjAgcHJpbyAyIHUzMiBodCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4eC41My4w LzI0IGhhc2hrZXkgbWFzayAweDAwMDAwMGZmIGF0IDEyIGxpbmsgMToKdGMgZmlsdGVyIGFkZCBk ZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50IDExOjAgcHJpbyAyIHUzMiBodCA4MDA6OiBt YXRjaCBpcCBzcmMgeHh4Lnh4eC41NC4wLzI0IGhhc2hrZXkgbWFzayAweDAwMDAwMGZmIGF0IDEy IGxpbmsgMToKdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50IDEx OjAgcHJpbyAyIHUzMiBodCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4eC41NS4wLzI0IGhhc2hr ZXkgbWFzayAweDAwMDAwMGZmIGF0IDEyIGxpbmsgMToKdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4 OTEgcHJvdG9jb2wgaXAgcGFyZW50IDExOjAgcHJpbyAyIHUzMiBodCA4MDA6OiBtYXRjaCBpcCBz cmMgeHh4Lnh4eC41Ni4wLzI0IGhhc2hrZXkgbWFzayAweDAwMDAwMGZmIGF0IDEyIGxpbmsgMToK dGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50IDExOjAgcHJpbyAy IHUzMiBodCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4eC41Ny4wLzI0IGhhc2hrZXkgbWFzayAw eDAwMDAwMGZmIGF0IDEyIGxpbmsgMToKdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9j b2wgaXAgcGFyZW50IDExOjAgcHJpbyAyIHUzMiBodCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4 eC41OC4wLzI0IGhhc2hrZXkgbWFzayAweDAwMDAwMGZmIGF0IDEyIGxpbmsgMToKdGMgZmlsdGVy IGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50IDExOjAgcHJpbyAyIHUzMiBodCA4 MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4eC41OS4wLzI0IGhhc2hrZXkgbWFzayAweDAwMDAwMGZm IGF0IDEyIGxpbmsgMToKdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFy ZW50IDExOjAgcHJpbyAyIHUzMiBodCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4eC42MC4wLzI0 IGhhc2hrZXkgbWFzayAweDAwMDAwMGZmIGF0IDEyIGxpbmsgMToKdGMgZmlsdGVyIGFkZCBkZXYg dmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50IDExOjAgcHJpbyAyIHUzMiBodCA4MDA6OiBtYXRj aCBpcCBzcmMgeHh4Lnh4eC42MS4wLzI0IGhhc2hrZXkgbWFzayAweDAwMDAwMGZmIGF0IDEyIGxp bmsgMToKdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50IDExOjAg cHJpbyAyIHUzMiBodCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4eC42Mi4wLzI0IGhhc2hrZXkg bWFzayAweDAwMDAwMGZmIGF0IDEyIGxpbmsgMToKdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEg cHJvdG9jb2wgaXAgcGFyZW50IDExOjAgcHJpbyAyIHUzMiBodCA4MDA6OiBtYXRjaCBpcCBzcmMg eHh4Lnh4eC42My4wLzI0IGhhc2hrZXkgbWFzayAweDAwMDAwMGZmIGF0IDEyIGxpbmsgMToKdGMg ZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAgcGFyZW50IDExOjAgcHJpbyAyIHUz MiBodCA4MDA6OiBtYXRjaCBpcCBzcmMgeHh4Lnh4eC42NC4wLzI0IGhhc2hrZXkgbWFzayAweDAw MDAwMGZmIGF0IDEyIGxpbmsgMToKCgppPTQ4OwpmbG93X2lkPTQ7Cgp3aGlsZSBbICRpIC1sZSA2 NCBdOyBkbwoJaj0wOwoJd2hpbGUgWyAkaiAtbGUgMjU1IF07IGRvCgkJaGV4PWBlY2hvICJvYmFz ZT0xNjsgJGoiIHxiY2A7CgkJdGMgZmlsdGVyIGFkZCBkZXYgdmxhbjA4OTEgcHJvdG9jb2wgaXAg cGFyZW50IDExOjAgcHJpbyA0IHUzMiBodCAxOiRoZXg6IG1hdGNoIGlwIHNyYyB4eHgueHh4LiRp LiRqIGZsb3dpZCAxMTokZmxvd19pZAoJCXRjIGNsYXNzIGFkZCBkZXYgdmxhbjA4OTEgcGFyZW50 IDExOjMgY2xhc3NpZCAxMTokZmxvd19pZCBoZnNjIHJ0IG0yIDFrYml0IGxzIGQgNW1zIG0yIDJr Yml0IHVsIG0yIDI1NmtiaXQgCgkJdGMgcWRpc2MgYWRkIGRldiB2bGFuMDg5MSBwYXJlbnQgMTE6 JGZsb3dfaWQgZXNmcSBxdWFudHVtIDE1MDAgcGVydHVyYiA2MCBoYXNoIGRzdCBsaW1pdCAxNgoK CQlqPSQoKGorMSkpOwoJCWZsb3dfaWQ9JCgoZmxvd19pZCsxKSk7Cglkb25lCgoJaT0kKChpKzEp KTsKZG9uZQo --=__PartD5F63A69.0__Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc --=__PartD5F63A69.0__=--
Konrad wrote:> > Szymon Miotk wrote: > > > I have P4 3.0 GHz, 1 GB RAM. > > I have 3500 potential users (top load about 800 users, average 400). I > > have 3 interfaces (2 WAN + 1 LAN), so I have 10500 queues total (3500 on > > each interface). > > The traffic is 24Mbit max, average 20Mbit. > > OK... What''s wrong? > > I can only use 4775 filters :/Konrad, Your script must specify a prio. It cannot be 0 and all filters must be given the same value; I use ''prio 5''. Google "LARTC please document this" for the tail end of the thread from which this comes. It will NEVER get fixed, I think. -- gypsy
gypsy wrote:> Your script must specify a prio. It cannot be 0 and all filters must be > given the same value; I use ''prio 5''.OK, but in my script sometimes I need diferent prio in filters. It''s very strange problem, because I haven''t always limited number of filters. Sometimes I can make more than 32k filters, in other time I can only 6000 :/ Maybe you right... but how can I use more filters, and different prio? -- Konrad Cempura /Lenthir/
Konrad wrote:> We have an error talking to the kernel > loops: 684 filters: 4788 classes: 2052 > > What''s wrong? > I need more filters :/ > > I have 2.6.11.11 kernel with new iproute2, u32 match mark support and > IMQ (AB)... > > Everyone can make theoretically 0xffff (65535) classes and qdiscs on one > device. And I think this is true, but I can''t add more filters then > 4775! :( > > --- > v=1; cnt=0; > tc qdisc add dev imq0 root handle 1:0 htb > > while : [ $v -le 11000 ]; doLoops for ever with the colon after while for me.> qu0=`printf "%x\n" $v` > > qu1=`printf "%x\n" $v`So qu0 = qu1 which makes> > tc filter add dev imq0 protocol ip parent 1:$qu0 pref 5 u32 match ip dst > 192.168.0.5 flowid 1:$qu1illogical. Andy.
I found thing which causes this problem. tc filter add dev imq1 protocol ip parent 2:0 pref 4 u32 match ip src ... match ip dst ... flowid 2:$q If parent is 2:0 then I can make many filers.... But if I use 2:x (other class, x is diffrent that root number) I''ll have only limited number of filters. You must set PRIO (= PREF)!... Will someone write patch? ;P It is very important problem! Filters in classes is being better working... (this is my opinion) when you have 5000 filters grouped in classes... (Or any volunteer to teach me how to write patches :P)