Hi there, I got a question about CBQ, hope anybody can help me (did not found anything in the archives). My setup is like this: all hosts are Athlon 800MHZ, 256 MByte RAM and 3com9x Netcards (100MBit) Distribution SuSE 7.0 -> Kernel 2.2.16 Host Setup: ---www_server_1 / --------------|-------------www_server_2 load balancer \ (with CBQ) ---www_server_3 192.168.10.17 all I want to do is shaping the INCOMING traffic this means that if I define a special IP only 200Kbit of HTTP request traffic (as an example) is forwarded to the webservers from that IP. The load balancer (Linux Virtual Server) works on IP basis and is integrated as a patch into the system-kernel. It distributes the packets via "direct routing" this means load balancer and www_server_X have all the same IP. If a package is received by the LB it changes the MAC Address of the package and forward it to the right www_server_X. The following attempts did not work: using the fw filter: implementing one of the following rules via ipchains did not work: (ip 192.168.10.15 is the client I want to restrict bandwidth) ipchains -A forward -p ip -d 192.168.10.17 m 1 -j ACCEPT or ipchains -A output -p ip -d 192.168.10.17 m 1 -j ACCEPT or ipchains -A forward -p ip -s 192.168.10.15 m 1 -j ACCEPT or ipchains -A output -p ip -s 192.168.10.15 m 1 -j ACCEPT the filter was set up with the following rule tc filter add dev eth0 protocol ip parent 100:0 prio 100 handle 1 fw classid 100:100 I far as I know with one of the first two rules the whole incoming traffic should be reduced to to let´s say 200Kbit, with the last two rules traffic from source IP 192.168.10.15 sould be reduced to 200Kbit. Non did work. using the u32 filter: tc filter add dev eth0 parent 100:0 protocol ip prio 100 u32 match ip src 192.168.10.15 flowid 100:100 did not work either I tried if CBQ really works with a setup like this client ---- www_server and an ipchains rule like ipchains -A output -p ip -d 192.168.10.15 m 1 -j ACCEPT the whole outgoing traffic was reduced to 200Kbit. So if anybody has an idea what I did wrong in first place I would be very happy if you could tell me. Or is it impossible to shape incomming traffic like this. If you need any further information please ask me. thanks, Joern P.S.: sorry for my bad english
On Mon, Oct 09, 2000 at 02:46:17PM +0200, joern maier wrote:> Hi there, > > I got a question about CBQ, hope anybody can help me (did not found > anything > in the archives).This is the first post ever on the LARTC list, so this does not amaze me :-)> My setup is like this: > all hosts are Athlon 800MHZ, 256 MByte RAM and 3com9x Netcards (100MBit) > Distribution SuSE 7.0 -> Kernel 2.2.16 > > Host Setup: > > ---www_server_1 > / > --------------|-------------www_server_2 > load balancer \ > (with CBQ) ---www_server_3 > 192.168.10.17 > > > all I want to do is shaping the INCOMING traffic this means > that if I define a special IP only 200Kbit of HTTP request > traffic (as an example) is forwarded to the webservers from > that IP.Well, you can''t shape incoming traffic directly. You can shape traffic going out to www_server_[123].> The load balancer (Linux Virtual Server) works on IP basis and > is integrated as a patch into the system-kernel. It distributes > the packets via "direct routing" this means load balancer and > www_server_X have all the same IP. If a package is received by > the LB it changes the MAC Address of the package and forward it > to the right www_server_X.Perhaps this interferes with Linux traffic shaping, not sure. Does your loadbalancer have multiple ethernet cards? If so, you could shape the ''backend card'' to limit itself to 200kbit.> The following attempts did not work: > > using the fw filter: > implementing one of the following rules via ipchains did not work: > (ip 192.168.10.15 is the client I want to restrict bandwidth) > > ipchains -A forward -p ip -d 192.168.10.17 m 1 -j ACCEPT > or > ipchains -A output -p ip -d 192.168.10.17 m 1 -j ACCEPT > or > ipchains -A forward -p ip -s 192.168.10.15 m 1 -j ACCEPT > or > ipchains -A output -p ip -s 192.168.10.15 m 1 -j ACCEPT > > the filter was set up with the following rule > > tc filter add dev eth0 protocol ip parent 100:0 prio 100 handle 1 fw > classid 100:100Did you enable ''shaping based on fwmark'' when compiling the kernel?> should be reduced to to let´s say 200Kbit, with the last two rules > traffic > from source IP 192.168.10.15 sould be reduced to 200Kbit. Non did work. > > using the u32 filter: > > tc filter add dev eth0 parent 100:0 protocol ip prio 100 u32 match ip > src 192.168.10.15 flowid 100:100Here you match outgoing traffic on eth0 with a source of your webbrowser client.> the whole outgoing traffic was reduced to 200Kbit. > So if anybody has an idea what I did wrong in first place I would be > very > happy if you could tell me. Or is it impossible to shape incomming > traffic > like this. If you need any further information please ask me.Please give some details on your network cards, and include where 192.168.10.15 is in this picture, and which card it is connected to, and which card the webservers are connected to. Regards, bert hubert -- PowerDNS Versatile DNS Services Trilab The Technology People ''SYN! .. SYN|ACK! .. ACK!'' - the mating call of the internet
bert hubert wrote:> > On Mon, Oct 09, 2000 at 04:32:58PM +0200, joern maier wrote: > > o.k. here some more details I haven´t mentioned yet > > Please keep it on the list, I don''t like to give private advice, I want > everyone to benefit.sorry -> I just pushed the reply button not thinking that it won´t get back to the list but to your private e-mail account> > > > > network cofiguration of the LB > > > > eth0 Link encap:Ethernet HWaddr 00:01:02:07:5F:CF > > inet addr:192.168.10.6 Bcast:192.168.255.255 > > Mask:255.255.255.0 > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > RX packets:50624 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:50630 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:100 > > Interrupt:11 Base address:0xe800 > > > > eth0:110 Link encap:Ethernet HWaddr 00:01:02:07:5F:CF > > inet addr:192.168.10.17 Bcast:192.168.255.255 > > Mask:255.255.255.255 > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > Interrupt:11 Base address:0xe800 > > > > lo Link encap:Local Loopback > > inet addr:127.0.0.2 Mask:255.255.255.0 > > UP LOOPBACK RUNNING MTU:3924 Metric:1 > > RX packets:77 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:77 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:0 > > > > the route table: > > > > Kernel IP routing table > > Destination Gateway Genmask Flags Metric Ref Use > > Iface > > lb.mynetwork.or * 255.255.255.255 UH 0 0 0 > > eth0 > > 192.168.10.0 * 255.255.255.0 U 0 0 0 > > eth0 > > default gw.mynetwork.or 0.0.0.0 UG 0 0 0 > > eth0 > > > > > > -> so I only got one ethernet card which is listening to the normal IP > > and a > > virtual IP (the LB IP) > > > > for the nodes behind the lb the setup looks like this > > > > eth0 Link encap:Ethernet HWaddr 00:01:02:07:60:29 > > inet addr:192.168.10.8 Bcast:192.168.10.255 > > Mask:255.255.255.0 > > UP BROADCAST NOTRAILERS RUNNING MTU:1500 Metric:1 > > RX packets:180379 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:183275 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:100 > > Interrupt:11 Base address:0xe800 > > > > lo Link encap:Local Loopback > > inet addr:127.0.0.1 Mask:255.0.0.0 > > UP LOOPBACK RUNNING MTU:3924 Metric:1 > > RX packets:77 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:77 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:0 > > > > lo:0 Link encap:Local Loopback > > inet addr:192.168.10.17 Mask:255.255.255.255 > > UP LOOPBACK RUNNING MTU:3924 Metric:1 > > > > routing table: > > Kernel IP routing table > > Destination Gateway Genmask Flags Metric Ref Use > > Iface > > lb.mynetwork.or * 255.255.255.255 UH 0 0 0 > > lo > > 192.168.10.0 * 255.255.255.0 U 0 0 0 > > eth0 > > loopback * 255.0.0.0 U 0 0 0 > > lo > > default gw.mynetwork.or 0.0.0.0 UG 0 0 0 > > eth0 > > > > > > [...snip] > > > > > > > > Well, you can''t shape incoming traffic directly. You can shape traffic going > > > out to www_server_[123]. > > > > > that´s what I wanted to do > > > > [...snip...] > > > > > > > > Did you enable ''shaping based on fwmark'' when compiling the kernel? > > > > > > > did not do that in first place -> but I recompiled the kernel right now > > and > > it didn´t work either. > > > > [...snip...] > > > > > Please give some details on your network cards, and include where > > > 192.168.10.15 is in this picture, and which card it is connected to, and > > > which card the webservers are connected to. > > > > > more detailed host setup > > > > > > ---www_server_1 (lo:0 192.168.10.17) > > / > > client --------------|-------------------www_server_2 (lo:0 > > 192.168.10.17) > > 192.168.10.15 load balancer \ > > (with CBQ) ---www_server_3 (lo:0 192.168.10.17) > > eth0:110 = 192.168.10.17 > > > > > > I tried to configure CBQ on the LB like this: > > # tc filter add dev eth0:110 protocol ip parent 100:0 prio 100 handle 1 > > fw classid 100:100 > > answer was: > > # Cannot find device eth0:110 > > > > does this mean that CBQ and virtual IP addresses do not work together ? > > > > ------- > > bert hubert wrote: > > > > On Mon, Oct 09, 2000 at 02:46:17PM +0200, joern maier wrote: > > > Hi there, > > > > > > I got a question about CBQ, hope anybody can help me (did not found > > > anything > > > in the archives). > > > > This is the first post ever on the LARTC list, so this does not amaze me > > :-) > > > > > My setup is like this: > > > all hosts are Athlon 800MHZ, 256 MByte RAM and 3com9x Netcards (100MBit) > > > Distribution SuSE 7.0 -> Kernel 2.2.16 > > > > > > Host Setup: > > > > > > ---www_server_1 > > > / > > > --------------|-------------www_server_2 > > > load balancer \ > > > (with CBQ) ---www_server_3 > > > 192.168.10.17 > > > > > > > > > all I want to do is shaping the INCOMING traffic this means > > > that if I define a special IP only 200Kbit of HTTP request > > > traffic (as an example) is forwarded to the webservers from > > > that IP. > > > > Well, you can''t shape incoming traffic directly. You can shape traffic > > going > > out to www_server_[123]. > > > > > The load balancer (Linux Virtual Server) works on IP basis and > > > is integrated as a patch into the system-kernel. It distributes > > > the packets via "direct routing" this means load balancer and > > > www_server_X have all the same IP. If a package is received by > > > the LB it changes the MAC Address of the package and forward it > > > to the right www_server_X. > > > > Perhaps this interferes with Linux traffic shaping, not sure. Does your > > loadbalancer have multiple ethernet cards? If so, you could shape the > > ''backend card'' to limit itself to 200kbit. > > > > > The following attempts did not work: > > > > > > using the fw filter: > > > implementing one of the following rules via ipchains did not work: > > > (ip 192.168.10.15 is the client I want to restrict bandwidth) > > > > > > ipchains -A forward -p ip -d 192.168.10.17 m 1 -j ACCEPT > > > or > > > ipchains -A output -p ip -d 192.168.10.17 m 1 -j ACCEPT > > > or > > > ipchains -A forward -p ip -s 192.168.10.15 m 1 -j ACCEPT > > > or > > > ipchains -A output -p ip -s 192.168.10.15 m 1 -j ACCEPT > > > > > > the filter was set up with the following rule > > > > > > tc filter add dev eth0 protocol ip parent 100:0 prio 100 handle 1 fw > > > classid 100:100 > > > > Did you enable ''shaping based on fwmark'' when compiling the kernel? > > > > > should be reduced to to let´s say 200Kbit, with the last two rules > > > traffic > > > from source IP 192.168.10.15 sould be reduced to 200Kbit. Non did work. > > > > > > using the u32 filter: > > > > > > tc filter add dev eth0 parent 100:0 protocol ip prio 100 u32 match ip > > > src 192.168.10.15 flowid 100:100 > > > > Here you match outgoing traffic on eth0 with a source of your webbrowser > > client. > > > > > the whole outgoing traffic was reduced to 200Kbit. > > > So if anybody has an idea what I did wrong in first place I would be > > > very > > > happy if you could tell me. Or is it impossible to shape incomming > > > traffic > > > like this. If you need any further information please ask me. > > > > Please give some details on your network cards, and include where > > 192.168.10.15 is in this picture, and which card it is connected to, and > > which card the webservers are connected to. > > > > Regards, > > > > bert hubert > > > > -- > > PowerDNS Versatile DNS Services > > Trilab The Technology People > > ''SYN! .. SYN|ACK! .. ACK!'' - the mating call of the internet > > > > _______________________________________________ > > LARTC mailing list / LARTC@mailman.ds9a.nl > > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: > > http://ds9a.nl/2.4Routing/ > > > > -- > PowerDNS Versatile DNS Services > Trilab The Technology People > ''SYN! .. SYN|ACK! .. ACK!'' - the mating call of the internet
bert hubert wrote:> > On Tue, Oct 10, 2000 at 12:36:36PM +0200, joern maier wrote: > > bert hubert wrote: > > > > > > On Mon, Oct 09, 2000 at 04:32:58PM +0200, joern maier wrote: > > > > o.k. here some more details I haven´t mentioned yet > > > > > > Please keep it on the list, I don''t like to give private advice, I want > > > everyone to benefit. > > > > sorry -> I just pushed the reply button not thinking that it won´t get > > back > > to the list but to your private e-mail account > > What we need to do is setup a class for packets going out on eth0 with a dst > address of the backend, and then limit that. CBQ doesn''t know what an > eth0:110 is, it only knows that you have eth0, and that addresses have been > assigned to it. Try running "ip addr show dev eth0", and you''ll see > 192.168.10.17 with eth0, and not with eth0:110. >I did run "ip addr show dev eth0" the result was this: 2: eth0: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc cbq qlen 100 link/ether 00:01:02:07:5f:cf brd ff:ff:ff:ff:ff:ff inet 192.168.10.6/24 brd 192.168.255.255 scope global eth0 inet 192.168.10.17/32 brd 192.168.255.255 scope global eth0:110 so to me it looks like ..10.17 works with eth0:110> You can either mark packets leaving your host with the destination of your > magic ip addresses, and you''ll limit them all together to 200kbit, or you > can try to make more classes, one for each backend server, and select on the > basis of the real mac address. > > Regards, > > bert hubert > > -- > PowerDNS Versatile DNS Services > Trilab The Technology People > ''SYN! .. SYN|ACK! .. ACK!'' - the mating call of the internet > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
On Tue, Oct 10, 2000 at 12:36:36PM +0200, joern maier wrote:> bert hubert wrote: > > > > On Mon, Oct 09, 2000 at 04:32:58PM +0200, joern maier wrote: > > > o.k. here some more details I haven´t mentioned yet > > > > Please keep it on the list, I don''t like to give private advice, I want > > everyone to benefit. > > sorry -> I just pushed the reply button not thinking that it won´t get > back > to the list but to your private e-mail accountWhat we need to do is setup a class for packets going out on eth0 with a dst address of the backend, and then limit that. CBQ doesn''t know what an eth0:110 is, it only knows that you have eth0, and that addresses have been assigned to it. Try running "ip addr show dev eth0", and you''ll see 192.168.10.17 with eth0, and not with eth0:110. You can either mark packets leaving your host with the destination of your magic ip addresses, and you''ll limit them all together to 200kbit, or you can try to make more classes, one for each backend server, and select on the basis of the real mac address. Regards, bert hubert -- PowerDNS Versatile DNS Services Trilab The Technology People ''SYN! .. SYN|ACK! .. ACK!'' - the mating call of the internet