LARTC - Apr 2007 - Prioritizing based on HTTP Content-Type header

I''m setting up a reverse-proxy on a limited-bandwidth pipe. The
system is Debian "etch" on Linux 2.6, using squid as the proxy.

As we''ve only got 5mbit to play with, what I''d really like to
do is
set up priority levels based on the Content-Type of the (outgoing)
response:

  1. text/* gets highest priority (along with
     application/x-javascript).
  2. image/* gets middle priority.
  3. */* gets lowest priority.

Today I tried just using tc, with netfilter''s "string" match
module
to select matching packets, with limited success: while it does
match the packet containing the response header, additional packets
in the same stream don''t retain the fwmark (unsurprisingly).

Does anyone have any ideas of -- or even better, experience with --
a stack which can achieve this? squid''s built-in rate limiting
doesn''t have the concept of borrowing bandwidth, so that''s
out.

I''m open to pretty much anything: userspace proxies (either in front
of or replacing squid) are fine.

Another option is simply to "punish" bandwidth hogs: the primary
goal is to ensure downloads of large files don''t slow down users
that are browing webpages. Possibly just using SFQ will work for
this, but I''m not sure.

Any suggestions would be appreciated. I''m even open to changing
platform (e.g. FreeBSD), but I''d prefer to stick with Debian as
it''s
what I''m most comfortable with.

You can use STRING + CONSAVE modules !!

mark packets...

because string match only "starter packet" ... the others packets from
the same connection isn´t marked.. consave can track this..

-j CONNMARK --restore-mark
-m string --string ''string'' --algo bm -j MARK --set-mark 1
-m string --string ''string2'' --algo bm -j MARK --set-mark 2
-m mark --mark 1 -j CONNMARK --save-mark
-m mark --mark 2 -j CONNMARK --save-mark


Michael Alger wrote:
> I''m setting up a reverse-proxy on a limited-bandwidth pipe. The
> system is Debian "etch" on Linux 2.6, using squid as the proxy.
>
> As we''ve only got 5mbit to play with, what I''d really
like to do is
> set up priority levels based on the Content-Type of the (outgoing)
> response:
>
>   1. text/* gets highest priority (along with
>      application/x-javascript).
>   2. image/* gets middle priority.
>   3. */* gets lowest priority.
>
> Today I tried just using tc, with netfilter''s "string"
match module
> to select matching packets, with limited success: while it does
> match the packet containing the response header, additional packets
> in the same stream don''t retain the fwmark (unsurprisingly).
>
> Does anyone have any ideas of -- or even better, experience with --
> a stack which can achieve this? squid''s built-in rate limiting
> doesn''t have the concept of borrowing bandwidth, so
that''s out.
>
> I''m open to pretty much anything: userspace proxies (either in
front
> of or replacing squid) are fine.
>
> Another option is simply to "punish" bandwidth hogs: the primary
> goal is to ensure downloads of large files don''t slow down users
> that are browing webpages. Possibly just using SFQ will work for
> this, but I''m not sure.
>
> Any suggestions would be appreciated. I''m even open to changing
> platform (e.g. FreeBSD), but I''d prefer to stick with Debian as
it''s
> what I''m most comfortable with.
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>   

-- 
Sds.

Alexandre J. Correa
Onda Internet
www.ondainternet.com.br
Linux User ID #142329

On Tue, Apr 24, 2007 at 06:13:37AM -0300, Alexandre J. Correa - Onda Internet
wrote:
> You can use STRING + CONSAVE modules !!
> 
> mark packets...
> 
> because string match only "starter packet" ... the others packets
from
> the same connection isn´t marked.. consave can track this..
> 
> -j CONNMARK --restore-mark
> -m string --string ''string'' --algo bm -j MARK --set-mark
1
> -m string --string ''string2'' --algo bm -j MARK --set-mark
2
> -m mark --mark 1 -j CONNMARK --save-mark
> -m mark --mark 2 -j CONNMARK --save-mark
I haven''t fully tested the shaping setup, but it appears to be
classifying packets correctly.

One limitation is that it can''t cope with SSL; fortunately
that''s
not a current requirement for us, but I probably will need to find a
solution for that at some point.

Anyway, thanks again.