Hi list... I work for a school in the netherlands with a 2mbit Internet uplink and about 3800 eager student who want to play games on the Internet using one of our 800 workstations. Problem was that those game playing students are concentrated in 2 of our 6 physical locations... and they consumed the bandwidth which the other location would like to use for educational purposes. The thing we did first was use squid... with success. The hit ratio on data transfer is 25-30%... "free" bandwidth. Today I took the plunge and started to use HTB traffic shaping... and (to my surprise) I got it going without much troubles. The setup I have chosen first divides the load over two classes: - one for Internet rate 2mbit and a 2mbit ceil - a second for our DMZ rate 98mbit and a 100mbit ceil Next I sub-classed the Internet bucket into 6 classes each with a 333kbit rate and a 2mbit ceil. This has had the effect that my DMZ can be accessed at full speed while they fairly share the Internet uplink. And the way it looks now it works :D Hail to all those people who wrote those fine docs _o_ This is enough reason to address this list... just to say "Thank you!", but there is more. At the moment I do not max out my Internet link... reason for this is I guess the squid proxy... The way it works now is that I have 2 types of filters in effect: - The DMZ: all packages with a src ip from my DMZ go to the big 98/100 bucket. - The Internet: all packages with a dst ip in one of our 6 networks gets placed in one of the 6 333/2000 buckets. But there is of course a src of packages I do not catch this way... and these are the squid cache hits. Because I filter on destination the cache hits get treated the same as cache misses. But cache hits are in effect local traffic... they do not originate from the Internet. So here (finally) the question.. Is there a way to identify cache hits from misses? I took a look at the advanced filtering chapter of course, but am really dazzled by that (and I thought I understood TCP/IP a bit ;)). Some further info that would perhaps help is that squid is run as a transparant proxy on the router/firewall. regards Peter Kaagman -- "His great aim was to escape from civilization, and, as soon as he had money, he went to Southern California."
I haven''t tried this myself, but I see l7filter (l7-filter.sf.net) has patterns for matching cache hits and misses... This may just work for your application? -justin Peter Kaagman wrote:> Hi list... > > I work for a school in the netherlands with a 2mbit Internet uplink and > about 3800 eager student who want to play games on the Internet using > one of our 800 workstations. > > Problem was that those game playing students are concentrated in 2 of > our 6 physical locations... and they consumed the bandwidth which the > other location would like to use for educational purposes. > > The thing we did first was use squid... with success. The hit ratio on > data transfer is 25-30%... "free" bandwidth. > > Today I took the plunge and started to use HTB traffic shaping... and > (to my surprise) I got it going without much troubles. > > The setup I have chosen first divides the load over two classes: > - one for Internet rate 2mbit and a 2mbit ceil > - a second for our DMZ rate 98mbit and a 100mbit ceil > > Next I sub-classed the Internet bucket into 6 classes each with a > 333kbit rate and a 2mbit ceil. > > This has had the effect that my DMZ can be accessed at full speed while > they fairly share the Internet uplink. > > And the way it looks now it works :D > Hail to all those people who wrote those fine docs _o_ > > This is enough reason to address this list... just to say "Thank you!", > but there is more. > > At the moment I do not max out my Internet link... reason for this is I > guess the squid proxy... > The way it works now is that I have 2 types of filters in effect: > - The DMZ: all packages with a src ip from my DMZ go to the big 98/100 > bucket. > - The Internet: all packages with a dst ip in one of our 6 networks > gets placed in one of the 6 333/2000 buckets. > > But there is of course a src of packages I do not catch this way... and > these are the squid cache hits. Because I filter on destination the cache > hits get treated the same as cache misses. But cache hits are in effect > local traffic... they do not originate from the Internet. > > So here (finally) the question.. > Is there a way to identify cache hits from misses? > > I took a look at the advanced filtering chapter of course, but am > really dazzled by that (and I thought I understood TCP/IP a bit ;)). > > Some further info that would perhaps help is that squid is run as a > transparant proxy on the router/firewall. > > regards > > Peter Kaagman
One of the ways is to use tos field, but I think you need to patch squid for it. Brief google search gave this: http://www.it-academy.bg/zph/ I''ve never used it though so I''m not sure if I can be of any more help on it. Hope this helps Martin On Thursday 26 May 2005 15:32, Peter Kaagman wrote:> Hi list... > > I work for a school in the netherlands with a 2mbit Internet uplink and > about 3800 eager student who want to play games on the Internet using > one of our 800 workstations. > > Problem was that those game playing students are concentrated in 2 of > our 6 physical locations... and they consumed the bandwidth which the > other location would like to use for educational purposes. > > The thing we did first was use squid... with success. The hit ratio on > data transfer is 25-30%... "free" bandwidth. > > Today I took the plunge and started to use HTB traffic shaping... and > (to my surprise) I got it going without much troubles. > > The setup I have chosen first divides the load over two classes: > - one for Internet rate 2mbit and a 2mbit ceil > - a second for our DMZ rate 98mbit and a 100mbit ceil > > Next I sub-classed the Internet bucket into 6 classes each with a > 333kbit rate and a 2mbit ceil. > > This has had the effect that my DMZ can be accessed at full speed while > they fairly share the Internet uplink. > > And the way it looks now it works :D > Hail to all those people who wrote those fine docs _o_ > > This is enough reason to address this list... just to say "Thank you!", > but there is more. > > At the moment I do not max out my Internet link... reason for this is I > guess the squid proxy... > The way it works now is that I have 2 types of filters in effect: > - The DMZ: all packages with a src ip from my DMZ go to the big 98/100 > bucket. > - The Internet: all packages with a dst ip in one of our 6 networks > gets placed in one of the 6 333/2000 buckets. > > But there is of course a src of packages I do not catch this way... and > these are the squid cache hits. Because I filter on destination the cache > hits get treated the same as cache misses. But cache hits are in effect > local traffic... they do not originate from the Internet. > > So here (finally) the question.. > Is there a way to identify cache hits from misses? > > I took a look at the advanced filtering chapter of course, but am > really dazzled by that (and I thought I understood TCP/IP a bit ;)). > > Some further info that would perhaps help is that squid is run as a > transparant proxy on the router/firewall. > > regards > > Peter Kaagman
Peter Kaagman wrote:>Hi list... > >I work for a school in the netherlands with a 2mbit Internet uplink and >about 3800 eager student who want to play games on the Internet using >one of our 800 workstations. > >Problem was that those game playing students are concentrated in 2 of >our 6 physical locations... and they consumed the bandwidth which the >other location would like to use for educational purposes. > >The thing we did first was use squid... with success. The hit ratio on >data transfer is 25-30%... "free" bandwidth. > >Today I took the plunge and started to use HTB traffic shaping... and >(to my surprise) I got it going without much troubles. > >The setup I have chosen first divides the load over two classes: >- one for Internet rate 2mbit and a 2mbit ceil >- a second for our DMZ rate 98mbit and a 100mbit ceil > >Next I sub-classed the Internet bucket into 6 classes each with a >333kbit rate and a 2mbit ceil. > >This has had the effect that my DMZ can be accessed at full speed while >they fairly share the Internet uplink. > >And the way it looks now it works :D >Hail to all those people who wrote those fine docs _o_ > >This is enough reason to address this list... just to say "Thank you!", >but there is more. > >At the moment I do not max out my Internet link... reason for this is I >guess the squid proxy... >The way it works now is that I have 2 types of filters in effect: >- The DMZ: all packages with a src ip from my DMZ go to the big 98/100 > bucket. >- The Internet: all packages with a dst ip in one of our 6 networks > gets placed in one of the 6 333/2000 buckets. > >But there is of course a src of packages I do not catch this way... and >these are the squid cache hits. Because I filter on destination the cache >hits get treated the same as cache misses. But cache hits are in effect >local traffic... they do not originate from the Internet. > >So here (finally) the question.. >Is there a way to identify cache hits from misses? > >I took a look at the advanced filtering chapter of course, but am >really dazzled by that (and I thought I understood TCP/IP a bit ;)). > >Some further info that would perhaps help is that squid is run as a >transparant proxy on the router/firewall. > >regards > >Peter Kaagman > >http://www.it-academy.bg/zph/
Peter Kaagman wrote:> Hi list...> So here (finally) the question..> Is there a way to identify cache hits from misses?there is a patch for squid available at http://www.it-academy.bg/zph/> Some further info that would perhaps help is that squid is run as a > transparant proxy on the router/firewall. > > regards > > Peter Kaagman-- dlupescu -- This message was scanned for spam and viruses by BitDefender. For more information please visit http://www.bitdefender.com/
On Thu, 26 May 2005 15:32:42 +0200 Peter Kaagman <p.kaagman@atlascollege.nl> wrote:>Problem was that those game playing students are concentrated in 2 of >our 6 physical locations... and they consumed the bandwidth which the >other location would like to use for educational purposes.Actually, according to my experience, online gaming requires only a little bandwidth (~3-5kB/s) but sustained over a longer period. Furthermore, if the latency jumps above ~200ms it becomes less playable, and above about 500ms it''s practically useless, so noone will be able to play anyway. IMHO your bandwidth is consumed by P2P applications or worms, which have a much more serious effect on this. I have a linux distribution (Route Hat) optimised for this type of application (many unrelated computers sharing the same line). It may help you and even if not directly, you can take some hints from the scripts. In fact several dormitories already use it to great satisfaction.>Peter KaagmanYours sincerely, Peter
On Thu, 26 May 2005 15:32:42 +0200 Peter Kaagman <p.kaagman@atlascollege.nl> wrote:> So here (finally) the question.. > Is there a way to identify cache hits from misses?Maybe I do not understend question, but I think it is very simple. There is option tcp_outgoing_address. Note it mean replacing source address. Then each user machine have it''s owne address. For this aim I added to dummy (may be any other) interface the addresses of another subnet. squid-2.5.STABLE9-1.100.6asp It''s working... But not working IMQ, that I need too. I do not understand strange intension to use only PREROUTING and POSTROUTING. If you will use IMQ you will need AB instead of default BA NAT -- С наилучшими пожеланиями, Панько Александр. With best regards, Pan''ko Alexander. pankoAA@yandex.ru http://interdon.net/~panko/ ICQ 231647363 XMMS playing nothing :-)
Peter Kaagman wrote:> But there is of course a src of packages I do not catch this way... and > these are the squid cache hits. Because I filter on destination the cache > hits get treated the same as cache misses. But cache hits are in effect > local traffic... they do not originate from the Internet.If squid is running on the same machine as your htb rules then (I think) the only way you can shape incoming traffic from the internet properly is to use imq. I have not used squid - so may be wrong, but the patches will let you classify hits so they can be let through at lan speed. But what about misses - I assume that squid will connect to the internet and fetch the data unlimited even if they then get served to the lan at restricted speed. Andy.
On Thu, 26 May 2005 19:56:09 +0100 Andy Furniss <andy.furniss@dsl.pipex.com> wrote:> Peter Kaagman wrote: > > > But there is of course a src of packages I do not catch this way... and > > these are the squid cache hits. Because I filter on destination the cache > > hits get treated the same as cache misses. But cache hits are in effect > > local traffic... they do not originate from the Internet. > > If squid is running on the same machine as your htb rules then (I think) > the only way you can shape incoming traffic from the internet properly > is to use imq.I think IMQ needed only if there are not one interface to shape.> > I have not used squid - so may be wrong, but the patches will let you > classify hits so they can be let through at lan speed. But what about > misses - I assume that squid will connect to the internet and fetch the > data unlimited even if they then get served to the lan at restricted speed. >Inet <-----(one for all src)NAT<--(MISSES)---(src 192.168.90.0/28 dst ''real Inet IP'')Squid<---(HITS+MISSES)---hosts Inet ----->(one for all dst)NAT--(MISSES)--->(dst 192.168.90.0/28 src ''real Inet IP'')Squid---(HITS+MISSES)--->hosts The last not fully right... But clenly illustrates the idea. You can simply shape the MISSES on one interface... -- С наилучшими пожеланиями, Панько Александр. With best regards, Pan''ko Alexander. pankoAA@yandex.ru http://interdon.net/~panko/ ICQ 231647363 XMMS playing nothing :-)
Pan''ko Alexander wrote:> On Thu, 26 May 2005 19:56:09 +0100 > Andy Furniss <andy.furniss@dsl.pipex.com> wrote: > > >>Peter Kaagman wrote: >> >> >>>But there is of course a src of packages I do not catch this way... and >>>these are the squid cache hits. Because I filter on destination the cache >>>hits get treated the same as cache misses. But cache hits are in effect >>>local traffic... they do not originate from the Internet. >> >>If squid is running on the same machine as your htb rules then (I think) >>the only way you can shape incoming traffic from the internet properly >>is to use imq. > > > I think IMQ needed only if there are not one interface to shape. > > >>I have not used squid - so may be wrong, but the patches will let you >>classify hits so they can be let through at lan speed. But what about >>misses - I assume that squid will connect to the internet and fetch the >>data unlimited even if they then get served to the lan at restricted speed. >> > > > > Inet <-----(one for all src)NAT<--(MISSES)---(src 192.168.90.0/28 dst ''real Inet IP'')Squid<---(HITS+MISSES)---hosts > > Inet ----->(one for all dst)NAT--(MISSES)--->(dst 192.168.90.0/28 src ''real Inet IP'')Squid---(HITS+MISSES)--->hosts > > The last not fully right... But clenly illustrates the idea. > > You can simply shape the MISSES on one interface... >Well remember I don''t use squid so don''t really know, but I imagine that all lan connections on the relevant ports go to squid and squid then makes seperate connections to the internet if required. So all traffic headed from the internet to squid will have the dst IP of the internet interface even if you hook imq after (de)NAT other traffic will have local dst addresses. Andy.
On Thu, May 26, 2005 at 07:56:09PM +0100, Andy Furniss wrote:> Peter Kaagman wrote: > > >But there is of course a src of packages I do not catch this way... and > >these are the squid cache hits. Because I filter on destination the cache > >hits get treated the same as cache misses. But cache hits are in effect > >local traffic... they do not originate from the Internet. > > If squid is running on the same machine as your htb rules then (I think) > the only way you can shape incoming traffic from the internet properly > is to use imq. >Not really sure what imq is, will have to look that up tomorrow.> I have not used squid - so may be wrong, but the patches will let you > classify hits so they can be let through at lan speed. But what about > misses - I assume that squid will connect to the internet and fetch the > data unlimited even if they then get served to the lan at restricted speed. > > Andy.I think you have a point there... atm I am only shaping at eth1, which is the LAN interface. I will still have to shape eth0, which is my Internet interface. Not only for squid to behave, but also to give my DMZ (and services on the LAN) a fighting change to connect when the link is full. But after shaping eth1, with all those classes for all the networks, shaping eth0 should be a piece of cake (I hope). But what I saw with iptraf was that eth1 "maxed out" at 2mbit while eth0 had a load of about 1.8mbit (in only maxed out on bursts). That was exactly what made me wonder where that difference came from. My best idea was that the difference was caused by the cache hits. But I think the squid zhp patch at it-academy.bg will solve my miss/hit problem. Taken from it''s documentation it does exactly what I want. Will try that tomorrow. Re-compiling squid is not really a problem... build the (slackware) package myself anyway since it did not come with the stock distro. After that I will probably have to look in to prioritising things like DNS, SMTP, IMAP and SSH. Although getting stuff out of our network is not really a problem, only the download is congested. Peter PS I found out that a couple of replies I made were actually private messages... sorry about that. Most of what was said it them is repeated in this message... except for the "Thanks for the quick reply" -- Frisbeetarianism, n.: The belief that when you die, your soul goes up on the roof and gets stuck.
On Thu, May 26, 2005 at 08:41:32PM +0100, Andy Furniss wrote:> Well remember I don''t use squid so don''t really know, but I imagine that > all lan connections on the relevant ports go to squid and squid then > makes seperate connections to the internet if required. So all traffic > headed from the internet to squid will have the dst IP of the internet > interface even if you hook imq after (de)NAT other traffic will have > local dst addresses.What I have seen is what happens on eth1... my LAN interface. I am able to use 2 kind of filters: - One kind on src ip: in this filter I check for my DMZ network and put the traffic in the LAN-speed class. - One kind on dst ip: in which I split up the rest of the traffic up for the 6 classes at Internet-speed This is because traffic on eth1 is de-NATed and squid spoofs the src ip of the original site. But again... this is just experience from testing it and deduction. And it seems to be working ;) If I am able to set the Tos field to a certain value for hits, I presume I can make a third kind of filter to put these hit-packages in the LAN-speed class. Peter -- Reisner''s Rule of Conceptual Inertia: If you think big enough, you''ll never have to do it.
tc filter add dev <dev> protocol ip parent 1:0 pref 1 u32 match ip dst
192.168.0.12 match ip sport 8080 0xffff match ip tos 8 0xff flowid 1:12
#If squid miss, traffic will go to user queue.
tc filter add dev imq1 protocol ip parent 1:0 pref 2 u32 match ip src
192.168.0.22 match ip sport 8080 0xffff flowid 1:30
#this catch all traffic from source port Squid... but with priority 2
and throw in LAN class.
I''m using this patch:
--- CUT ---
diff -cr squid-2.5.STABLE3/src/client_side.c
squid-2.5.STABLE3.patched/src/client_side.c
*** squid-2.5.STABLE3/src/client_side.c 2003-05-24 13:08:41.000000000 +0200
--- squid-2.5.STABLE3.patched/src/client_side.c 2003-08-05
22:08:15.000000000 +0200
***************
*** 2005,2010 ****
--- 2005,2013 ----
/* Avoid copying to MemBuf for non-range requests */
/* Note, if we''re here, then ''rep'' is known to be
NULL */
http->out.offset += body_size;
+ { int tos=isTcpHit(http->log_type) ? 0 : 8;
+ setsockopt(fd,SOL_IP,IP_TOS,&tos,4);
+ }
comm_write(fd, buf, size, clientWriteBodyComplete, http, NULL);
/* NULL because clientWriteBodyComplete frees it */
return;
***************
*** 2062,2067 ****
--- 2065,2073 ----
if (!http->request->range && http->request->method ==
METHOD_GET)
assert(check_size == size);
/* write */
+ { int tos=isTcpHit(http->log_type) ? 0 : 8;
+ setsockopt(fd,SOL_IP,IP_TOS,&tos,4);
+ }
comm_write_mbuf(fd, mb, clientWriteComplete, http);
/* if we don''t do it, who will? */
memFree(buf, MEM_CLIENT_SOCK_BUF);
--- CUT ---
If squid HIT then TOS is 0, if miss then 1.
Here is the place where you can download this:
http://sed.pl/~mrk/qos/squid_hit_miss_mark.patch
Throw in to queue of LAN only traffic coming from Squid.
Upload traffic from Squid always queue in users classes.
On Thursday 26 May 2005 11:32 pm, Peter Kaagman wrote: Hi Peter,> The way it works now is that I have 2 types of filters in effect: > - The DMZ: all packages with a src ip from my DMZ go to the big 98/100 > bucket. > - The Internet: all packages with a dst ip in one of our 6 networks > gets placed in one of the 6 333/2000 buckets. > > But there is of course a src of packages I do not catch this way... and > these are the squid cache hits. Because I filter on destination the cache > hits get treated the same as cache misses. But cache hits are in effect > local traffic... they do not originate from the Internet. > > So here (finally) the question.. > Is there a way to identify cache hits from misses?Another approach would be to make use of the delay pools feature in squid. As you are using a transparent proxy, you could allocate more to squid through HTB on the internal interface for cached material and clamp squid incl. each user/group according to an ACL on the inet interface. This way you can more finely control the total amount of bandwidth, bandwidth per user/group etc. is able to suck through your inet interface. You can''t easily discriminate groups using plain ol htb. e.g. in squid.conf something like delay_pools 2 # 2 delay pools delay_class 1 2 # pool 1 is a class 2 pool delay_class 2 2 # pool 2 is a class 2 pool delay_access 1 allow admins delay_access 1 deny all delay_access 2 allow our_networks delay_access 2 deny all delay_parameters 1 -1/-1 32000/24000 48000/15000 delay_parameters 2 -1/-1 24000/15000 24000/15000 See the squid.conf doc''s for a description. Cheers, Lewis
Peter Kaagman wrote:> On Thu, May 26, 2005 at 07:56:09PM +0100, Andy Furniss wrote: > >>Peter Kaagman wrote: >> >> >>>But there is of course a src of packages I do not catch this way... and >>>these are the squid cache hits. Because I filter on destination the cache >>>hits get treated the same as cache misses. But cache hits are in effect >>>local traffic... they do not originate from the Internet. >> >>If squid is running on the same machine as your htb rules then (I think) >>the only way you can shape incoming traffic from the internet properly >>is to use imq. >> > > > Not really sure what imq is, will have to look that up tomorrow.I think using delay pools like Lewis says could be another way. If you don''t have much traffic that needs priority over squid then you may be able to get away with shaping on lan facing eth with the same settings as the delay pools. If you use imq then you won''t be able to tell which user squid is fetching the data for. Whatever you do remember that shaping download is shaping traffic that has already been shaped by your link - so you need to back off from the link speed to have any chance of getting control, it still won''t be perfect if you care alot about latency. Andy.