LARTC - Jan 2006 - Per user bandwidth limiting ..for small ISP.using Squid

Sir,

 Kindly excuse me. I am a newbie to LARTC..

 I am a small ISP in rural India distributing 1 MB
link to 200 people.

I have been using rshaper  by Alessandro Rubini for
shaping.

http://freshmeat.net/projects/rshaper/

My kernel is Linux version 2.4.22-1.2115.nptl( Fedora
Core 1)

Rshaper is very good in controlling incoming bandwidth
(from LAN)

I use  Squid also on this Linux Box..

Right now I am using Delay Pools of Squid to control
bandwidth  per user for Http traffic.

Squid saves me around 35% of bandwidth and hence I can
not afford not to use it.

Squid also gives my clients a feel of speed --an
important thing for me.

Rshaper is no more under active development. The
author advised me to switch to TC..



Rshaper will not work on any new kernel..

I want to switch to kernel 2.6

Any Ideas for this kind of a situation  using tc ,
Squid..

Bandwidth of 1 MB for 200 users means  "tiny" rates I
have to use for each client (7-8 Kbit).

Using rshper I  wrote scripts

 rshaperctl     10.x.x.1       2400
     .      .    .    .
     .      .    .    .
     .      .    .    .

 rshaperctl     10.x.x.250      2400

which means the Clinet 10.x.x.1 sends  @2400 bytes/sec
and gets 3 times 2400(7200) downloads..( the default
behaviuor of rshaper)


(all 250 lines ..batch file) and for port 80 (Squid) I
am using delay pools ..

I want browsing should be fast .. downloads can be slow..

I want a similar solution using TC , Htb ..

Kindly Help...

Thankx a lot for your time....

Rayudu.


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Madhava Rayudu schrieb:
> Sir,
Hello,
>  I am a small ISP in rural India distributing 1 MB
> link to 200 people.
...
> Squid saves me around 35% of bandwidth and hence I can
> not afford not to use it.

I hope people won''t mind if I mention my project again:
http://www.shurdix.org

For some time it was successfully used on a network with 8MBit link and 
1400 users (although the peak of simultaneously active users was
"only"
slightly above 700). So it should be able to work in your situation.

Your situation is however special because you have squid. Combining 
squid and tc is problematic. However, there were some kind guys who 
designed the "tproxy" iptables extension, which can help you. It
isn''t
easy to setup and if you have NAT you need 2 separate machines (one 
doing the NAT and one running the squid), but is doable. This way tc 
will see squid''s traffic with the IP of the real client. Squid
including
the tproxy patch can be found in the optional package for shurdix (I''ll
document it on the wiki page if you are interested).

My recommendation for your situation would be something like this:
- keep your router, let it do NAT and perhaps a minimal firewall
- get a second machine, put it between the router and the LAN, and 
install shurdix there
- configure it to use TC and Squid (and optionally IP accounting and/or 
firewall if you like). No delay pools necessary.

I would like to stress again however that combining tproxied squid and 
tc isn''t easy and is poorly documented (but possible and it works).
YMMV.
> Rayudu.
Yours sincerely,
Peter

-- 
http://www.shurdix.org - Linux distribution for routers and firewalls

On Fri, Jan 13, 2006 at 04:58:19PM +0100, Peter Surda
wrote:
> I hope people won''t mind if I mention my project again:
> http://www.shurdix.org
We''re happy to receive any reply at all, really... :-)
> Your situation is however special because you have squid. Combining 
> squid and tc is problematic.
I agree; so far I haven''t been able to shape squid traffic 
the way I want it to. However, shouldn''t rshaper suffer from the 
same issues? It should at least be possible to do something 
similar to rshaper using tc.
> However, there were some kind guys who designed the "tproxy"
iptables
> extension, which can help you. It isn''t easy to setup and if you
have
> NAT you need 2 separate machines (one doing the NAT and one running 
> the squid), but is doable. This way tc will see squid''s traffic
with
> the IP of the real client.
These are about the most interesting lines I''ve seen on this topic. 
However, I''m in a small home network situation, so even having just 
one dedicated linux machine is luxury. So any solution that requires 
separate machines is not feasible for me.
> My recommendation for your situation would be something like this:
> - keep your router, let it do NAT and perhaps a minimal firewall
> - get a second machine, put it between the router and the LAN, and 
> install shurdix there
> - configure it to use TC and Squid (and optionally IP accounting and/or 
> firewall if you like). No delay pools necessary.
Other possibilities are:
- Never touch a running system. (If it works, why not leave as is?)
- Find out how exactly rshaper limits and/or distributes
  up- and download bandwidth for
    * User <-> Internet
    * User <-> User
    * Internet <-> Squid (and other caches, DNS etc.)
    * Squid (and others?) <-> User
  and use this information to build a tc class tree.
- If you want to keep rshaper, port it to 2.6 by yourself ;-)

Regards,
Andreas Klauer

Andreas Klauer schrieb:
>>However, there were some kind guys who designed the "tproxy"
iptables
>>extension, which can help you.
>>
(cut)
>These are about the most interesting lines I''ve seen on this topic.
>However, I''m in a small home network situation, so even having just
>one dedicated linux machine is luxury. So any solution that requires 
>separate machines is not feasible for me.
>  
>
Unfortunately for design reasons, TPROXY and NAT won''t work together
and
AFAIK there are no plans to change this. I didn''t investigate deeply, 
but I assume TPROXY uses the fields reserved for NAT for other purposes. 
So if you need both NAT and TPROXY, you need 2 boxes (and some hacking 
with the routing or arptables or both ;-)).
>Other possibilities are:
>- Never touch a running system. (If it works, why not leave as is?)
>  
>
Actually this is a great idea. I admit I didn''t read the original post 
completely and assumed that a new system is required for some reason.
>- Find out how exactly rshaper limits and/or distributes
>  
>
Upon looking at the docs for rshaper, I don''t think it distributes 
anything (only limits and has no borrowing). This can be done with HTB 
(and IMQ). Several years ago I wrote a bandwidth management system for a 
small ISP that actually worked somewhat like this (the ISP uses a web 
interface to set incoming/outgoing bandwith for individual customers, 
and optionally a monthly limit, and cron sets up the HTB rules 
automagically). I don''t use it personally, Shurdix does fair 
distribution only, but I imagine there are people who might have other 
requirements. If there is enough interest (and I find the time) I can 
polish it and put it for download.
>Regards,
>Andreas Klauer
>  
>
Yours sincerely,
Peter

-- 
http://www.shurdix.org - Linux distribution for routers and firewalls

Upon looking at the docs for rshaper, I don''t think it
distributes
> anything (only limits and has no borrowing). This can be done with HTB
> (and IMQ). Several years ago I wrote a bandwidth management system for a
> small ISP that actually worked somewhat like this (the ISP uses a web
> interface to set incoming/outgoing bandwith for individual customers,
> and optionally a monthly limit, and cron sets up the HTB rules
> automagically). I don''t use it personally, Shurdix does fair
> distribution only, but I imagine there are people who might have other
> requirements. If there is enough interest (and I find the time) I can
> polish it and put it for download.


My requiremet suits exactly this..
Kindly post t it I will Download ...
Thanx a lot for great help..


Regards,
Rayudu


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Thanks a lot  ...Andreas Klauer


Upon looking at the docs for rshaper, I don''t think it
distributes
> anything (only limits and has no borrowing). This can be done with HTB
> (and IMQ). Several years ago I wrote a bandwidth management system for a
> small ISP that actually worked somewhat like this (the ISP uses a web
> interface to set incoming/outgoing bandwith for individual customers,
> and optionally a monthly limit, and cron sets up the HTB rules
> automagically). I don''t use it personally, Shurdix does fair
> distribution only, but I imagine there are people who might have other
> requirements. If there is enough interest (and I find the time) I can
> polish it and put it for download.


This is what I want... Make it accessible .. I will download ..

Rayudu.


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Madhava Rayudu schrieb:
> This is what I want... Make it accessible .. I will download ..
Please watch this thread for updates:
http://forum.shurdix.org/viewtopic.php?t=56

I can''t publish it now, it may contain private data that
doesn''t belong
to me. Wait a couple of weeks pls.
> Rayudu.
Yours sincerely,
Peter

-- 
http://www.shurdix.org - Linux distribution for routers and firewalls