LARTC - Jun 2005 - Help? Router/Bandwidth throttle needed.

I hope this list is still active.  I''m an experienced Linux Sysadmin,
but I
haven''t done much in the way of routing.  Due to a decision made by my
higherups, I need to jam a computer between my ISP and my LAN to do
bandwidth throttling.

My current setup:

1 Crappy Cable Modem (7Mb/768Kb connection) with a static IP.
4 servers (all have static, routable IPs) - One of which is running NAT/DHCP
(2 NICs) for 6 workstations (All using DHCP)

I have 1 spare routable IP.

My current setup has the cable modem plugged directly into my LAN.  But it
ends up this modem doesn''t do intelligent traffic/bandwidth management.
I
need a way to throttle and poll usage through the modem such that neither
uploads nor downloads will ever use the full capacity (and thus dominate the
router).

Currently, the best I can do, is to throttle some of those 4 servers, but
that doesn''t help me with the workstations.   So I figured I''d
need to jam a
machine between the modem and the LAN to work as a
''gatekeeper'' to control
the bidirectional traffic (and I could use it for a firewall too).

Can someone help me on how I need to set this up?  I have a spare box I
could easily put two NICs into, if needed, to make it the gatekeeper.

-----

Michael Yacht
CTO
Ideal Conditions, Inc.
5329 Beeler Street, #2
Pittsburgh, PA 15217
v: 412-325-1375

ICI Support wrote:
> 
> I hope this list is still active.  I''m an experienced Linux
Sysadmin, but I
> haven''t done much in the way of routing.  Due to a decision made
by my
> higherups, I need to jam a computer between my ISP and my LAN to do
> bandwidth throttling.
> 
> My current setup:
> 
> 1 Crappy Cable Modem (7Mb/768Kb connection) with a static IP.
> 4 servers (all have static, routable IPs) - One of which is running
NAT/DHCP
> (2 NICs) for 6 workstations (All using DHCP)
> 
> I have 1 spare routable IP.
> 
> My current setup has the cable modem plugged directly into my LAN.  But it
> ends up this modem doesn''t do intelligent traffic/bandwidth
management.   I
> need a way to throttle and poll usage through the modem such that neither
> uploads nor downloads will ever use the full capacity (and thus dominate
the
> router).
> 
> Currently, the best I can do, is to throttle some of those 4 servers, but
> that doesn''t help me with the workstations.   So I figured
I''d need to jam a
> machine between the modem and the LAN to work as a
''gatekeeper'' to control
> the bidirectional traffic (and I could use it for a firewall too).
> 
> Can someone help me on how I need to set this up?  I have a spare box I
> could easily put two NICs into, if needed, to make it the gatekeeper.
> 
> -----
> 
> Michael Yacht
> CTO
> Ideal Conditions, Inc.
> 5329 Beeler Street, #2
> Pittsburgh, PA 15217
> v: 412-325-1375
Michael,

I have exactly your setup except that I have 2 internet connections and
(at long last) neither is crappy.

http://yesican.chsoft.biz/lartc

The top part of that describes my setup.  Stop when you get to "Now we
have 2 providers".

Short description:  eth0 points to the internal servers which have
external IPs and I shape the traffic going out on that interface.  eth1
points to the internet and I shape the traffic going out on that.  In
order to answer for the 4 internal machines, proxyarp is employed.  Each
of the 4 internal machines has a firewall, but these are a LOT quieter
since I put a firewall on this computer <grin>.

You may also want to ftp andthatsjazz.org/pub/lartc
--
gypsy

The webpage was helpful in some ways, but I think it is a step too advanced
for me at this time.

I guess what I want to know is if the ''plan'' I have is sound?

I want to put a linux box between my cable modem and my lan.  It''ll go
from
the cablemodem into eth0 then from eth1 to the LAN.

I just want to do complete passthrough (with packets intact) if possible
from one side to the other.  This machine doesn''t have to do NAT or
anything.

Then I want to throttle the bandwidth through that box intelligently (I kind
of know how to do that from the Advanced Routing Howto.  I''ve done it
on the
other three servers to throttle the bandwidth down).

Once all that is working, I''ll deal with using iptables to firewall.


If this plan is sound, I guess I need to know the vague steps involved to
accomplish the routing.  The throttling I can figure out myself.

-Michael 
> -----Original Message-----
> From: gypsy [mailto:gypsy@iswest.com]
> Sent: Monday, June 20, 2005 10:09 PM
> To: lartc@mailman.ds9a.nl
> Cc: ICI Support
> Subject: Re: [LARTC] Help? Router/Bandwidth throttle needed.
> 
> ICI Support wrote:
> >
> > I hope this list is still active.  I''m an experienced Linux
Sysadmin,
> but I
> > haven''t done much in the way of routing.  Due to a decision
made by my
> > higherups, I need to jam a computer between my ISP and my LAN to do
> > bandwidth throttling.
> >
> > My current setup:
> >
> > 1 Crappy Cable Modem (7Mb/768Kb connection) with a static IP.
> > 4 servers (all have static, routable IPs) - One of which is running
> NAT/DHCP
> > (2 NICs) for 6 workstations (All using DHCP)
> >
> > I have 1 spare routable IP.
> >
> > My current setup has the cable modem plugged directly into my LAN. 
But
> it
> > ends up this modem doesn''t do intelligent traffic/bandwidth
management.
> I
> > need a way to throttle and poll usage through the modem such that
> neither
> > uploads nor downloads will ever use the full capacity (and thus
dominate
> the
> > router).
> >
> > Currently, the best I can do, is to throttle some of those 4 servers,
> but
> > that doesn''t help me with the workstations.   So I figured
I''d need to
> jam a
> > machine between the modem and the LAN to work as a
''gatekeeper'' to
> control
> > the bidirectional traffic (and I could use it for a firewall too).
> >
> > Can someone help me on how I need to set this up?  I have a spare box
I
> > could easily put two NICs into, if needed, to make it the gatekeeper.
> >
> > -----
> >
> > Michael Yacht
> > CTO
> > Ideal Conditions, Inc.
> > 5329 Beeler Street, #2
> > Pittsburgh, PA 15217
> > v: 412-325-1375
> 
> Michael,
> 
> I have exactly your setup except that I have 2 internet connections and
> (at long last) neither is crappy.
> 
> http://yesican.chsoft.biz/lartc
> 
> The top part of that describes my setup.  Stop when you get to "Now we
> have 2 providers".
> 
> Short description:  eth0 points to the internal servers which have
> external IPs and I shape the traffic going out on that interface.  eth1
> points to the internet and I shape the traffic going out on that.  In
> order to answer for the 4 internal machines, proxyarp is employed.  Each
> of the 4 internal machines has a firewall, but these are a LOT quieter
> since I put a firewall on this computer <grin>.
> 
> You may also want to ftp andthatsjazz.org/pub/lartc
> --
> gypsy

> The webpage was helpful in some ways, but I think it is a step too 
advanced
> for me at this time.
> 
> I guess what I want to know is if the ''plan'' I have is
sound?
> 
> I want to put a linux box between my cable modem and my lan. 
It''ll go
from
> the cablemodem into eth0 then from eth1 to the LAN.
> 
> I just want to do complete passthrough (with packets intact) if possible
> from one side to the other.  This machine doesn''t have to do NAT
or
> anything.
> 
> Then I want to throttle the bandwidth through that box intelligently (I 
kind
> of know how to do that from the Advanced Routing Howto.  I''ve done
it on
the
> other three servers to throttle the bandwidth down).
> 
> Once all that is working, I''ll deal with using iptables to
firewall.
> 
> 
> If this plan is sound, I guess I need to know the vague steps involved 
to
> accomplish the routing.  The throttling I can figure out myself.
> 
> -Michael 
> 
Michael,

You''re on the right list. Your plan is very doable. A solution would be

--------     ---------    -------
| cable|-----| linux |----| LAN |
|modem |     | bridge|    -------
--------     | w/QoS |
             ---------

Just so happens I have a HOWTO on my website. You''ll need to tweak the 
shaping for your own needs. For your servers you could do a total max 
limit (so users still always get some bandwidth) and implement SFQ 
(fairness queuing) so that if multiple servers are getting hit, they 
divide the bandwidth up as fairly as possible. Using src and dest IP the 
servers could be isolated. Then you could lump all other traffic together. 
Or my preference is to create another class for say port 80, or any other 
apps, that is above the default (bulk traffic).

Here''s the HOWTO

http://www.burnpc.com/website.nsf/all/3a64a6369757819686256f960068ad75!OpenDocument


HTH,
-Ron

Here''s a ''map'' of my network
''plan'':

+-------+   +--------+          +-------+
| Dumb  |   | Linux  |          | Linux |
| Cable +---+ Bridge +----+-----+ DHCP/ +----> Bunch of workstations
| Modem |   | w/ QoS |    |     | NAT   |      on 192.168.10.X
+-------+   +--------+    |     +-------+
X.X.X.62    Eth0 X.X.X.61 |     Eth0 X.X.X.57 
            Eth1 ???      |     Eth1 192.168.10.1
                          |
                          |
           			  +-----> 3 more servers:  X.X.X.58->60

The X.X.X IPs are routable IPs, just didn''t feel like spamming you with
the
#s.

Can you walk me through how some of those configs you have (particularly
your bridging script) would work with this?  

While I''ve been dealing with Linux for about 13 years now, I
haven''t EVER
had to do any routing stuff, and this is really starting to stymie me. 
I''m
a programmer, not a sysadmin, by job description.  I''m just the closest
thing we have to a sysadmin (and good at all the non-advanced stuff).

Also, any reason I''d have to use FC?  I''m rather partial to
Slackware (only
because it is on lots of my machines already, and I like standardizing on 1
distro for my ease of management).  I know slackware has iptables, tc,
etc... since I''ve already enabled single-host throttling on 2 other
Linux
servers (.57 and .58 on my network).

-Michael 
> -----Original Message-----
> From: rsenykoff@harrislogic.com [mailto:rsenykoff@harrislogic.com]
> Sent: Wednesday, June 22, 2005 12:31 PM
> To: Support@idealconditions.com
> Cc: gypsy@iswest.com; lartc@mailman.ds9a.nl; lartc-bounces@mailman.ds9a.nl
> Subject: RE: [LARTC] Help? Router/Bandwidth throttle needed.
> 
> > The webpage was helpful in some ways, but I think it is a step too
> advanced
> > for me at this time.
> >
> > I guess what I want to know is if the ''plan'' I have
is sound?
> >
> > I want to put a linux box between my cable modem and my lan. 
It''ll go
> from
> > the cablemodem into eth0 then from eth1 to the LAN.
> >
> > I just want to do complete passthrough (with packets intact) if
possible
> > from one side to the other.  This machine doesn''t have to do
NAT or
> > anything.
> >
> > Then I want to throttle the bandwidth through that box intelligently
(I
> kind
> > of know how to do that from the Advanced Routing Howto.  I''ve
done it on
> the
> > other three servers to throttle the bandwidth down).
> >
> > Once all that is working, I''ll deal with using iptables to
firewall.
> >
> >
> > If this plan is sound, I guess I need to know the vague steps involved
> to
> > accomplish the routing.  The throttling I can figure out myself.
> >
> > -Michael
> >
> 
> Michael,
> 
> You''re on the right list. Your plan is very doable. A solution
would be
> 
> --------     ---------    -------
> | cable|-----| linux |----| LAN |
> |modem |     | bridge|    -------
> --------     | w/QoS |
>              ---------
> 
> Just so happens I have a HOWTO on my website. You''ll need to tweak
the
> shaping for your own needs. For your servers you could do a total max
> limit (so users still always get some bandwidth) and implement SFQ
> (fairness queuing) so that if multiple servers are getting hit, they
> divide the bandwidth up as fairly as possible. Using src and dest IP the
> servers could be isolated. Then you could lump all other traffic together.
> Or my preference is to create another class for say port 80, or any other
> apps, that is above the default (bulk traffic).
> 
> Here''s the HOWTO
> 
> http://www.burnpc.com/website.nsf/all/3a64a6369757819686256f960068ad75!Ope
> nDocument
> 
> 
> HTH,
> -Ron

> Here''s a ''map'' of my network
''plan'':
> 
> +-------+   +--------+          +-------+
> | Dumb  |   | Linux  |          | Linux |
> | Cable +---+ Bridge +----+-----+ DHCP/ +----> Bunch of workstations
> | Modem |   | w/ QoS |    |     | NAT   |      on 192.168.10.X
> +-------+   +--------+    |     +-------+
> X.X.X.62    Eth0 X.X.X.61 |     Eth0 X.X.X.57 
>             Eth1 ???      |     Eth1 192.168.10.1
>                           |
>                           |
>                       +-----> 3 more servers:  X.X.X.58->60
> 
> The X.X.X IPs are routable IPs, just didn''t feel like spamming you
with
the
> #s.
> 
> Can you walk me through how some of those configs you have (particularly
> your bridging script) would work with this? 
> 
> While I''ve been dealing with Linux for about 13 years now, I
haven''t
EVER
> had to do any routing stuff, and this is really starting to stymie me. 
I''m
> a programmer, not a sysadmin, by job description.  I''m just the
closest
> thing we have to a sysadmin (and good at all the non-advanced stuff).
> 
> Also, any reason I''d have to use FC?  I''m rather partial
to Slackware
(only
> because it is on lots of my machines already, and I like standardizing 
on 1
> distro for my ease of management).  I know slackware has iptables, tc,
> etc... since I''ve already enabled single-host throttling on 2
other
Linux
> servers (.57 and .58 on my network).
> 
Any linux distro with 2.4 or 2.6 kernel will include the traffic control 
stuff you need.

The individual NICs on the bridge don''t get IPs. The bridge itself can
be
assigned an IP however, for remote management.

Part of my bridging script deals with Spanning Tree Protocol, which you 
can leave alone (I use it for redundancy on the old boxes we use as 
bridges / QoS).

#whatever you want to call the bridge... not that important
BRIDGENAME="MyBridge1"

# IP of the gateway... the cable modem in your case
GATEWAY="X.X.X.X" 

# IP you want to remote admin the bridge from (.61)
BRIDGE_IP="X.X.X.X"

# whatever the netmask should be for your network
BRIDGE_NETMASK="X.X.X.X"

# unless you''re going to run multiple redundant bridges, change this to
"no"
STP_ON="yes"

# leave alone unless using STP in which case one bridge gets 1, the other 
2
STP_PRIO="1"

The script does assume that the NICs are eth0 and eth1. Change if needed. 
You should make a note of which NIC goes which way (using MAC address is 
good) and label on the back. This will come in handy as your upload is not 
the same as your download.

Comment these lines out until you get the bridge working, then we can help 
you get the shaping going.
sh /etc/rc.d/init.d/shape.eth0.sh
sh /etc/rc.d/init.d/shape.eth1.sh

This script is however kind of designed for Redhat with the chkconfig 
lines, start stop etc, although I know it works on Gentoo, so you''ll
need
to figure that one out. After 13 years of Linux administration I''m sure
you can do that.

-Ron

rsenykoff@harrislogic.com schrieb:
>>Here''s a ''map'' of my network
''plan'':
>>
>>+-------+   +--------+          +-------+
>>| Dumb  |   | Linux  |          | Linux |
>>| Cable +---+ Bridge +----+-----+ DHCP/ +----> Bunch of workstations
>>| Modem |   | w/ QoS |    |     | NAT   |      on 192.168.10.X
>>+-------+   +--------+    |     +-------+
>>X.X.X.62    Eth0 X.X.X.61 |     Eth0 X.X.X.57 
>>            Eth1 ???      |     Eth1 192.168.10.1
>>                          |
>>                          |
>>                      +-----> 3 more servers:  X.X.X.58->60
> [...]
> The script does assume that the NICs are eth0 and eth1. Change if needed. 
> You should make a note of which NIC goes which way (using MAC address is 
> good) and label on the back. This will come in handy as your upload is not 
> the same as your download.
Using ifrename to name your interfaces according to the function they have
is very much recommended. That way you never have to remember which
interface is the one you want to access. I use names like bridgeext for
eth0 and bridgeint for eth1.

Another recommendation for bridging is to use a separate interface for
administration and to assign no IP to br0/bridgeext/bridgeint. That way
the machine will still be accessible even if STP and friends go crazy
on the bridged nics.


Regards,
Carl-Daniel
-- 
http://www.hailfinger.org/

I want to thank everyone so much for their help, especially Peter Surda who
was invaluable as we traded email all day long.

I''ve got the bridge working, got good queuing going.  Now I''ll
work on the
firewall (which I can do myself.. hehe)

You guys are awesome.  If this works well I''ll be back in a month or so
as
we add a second cable modem to the pool and you can help me figure out how
to get some cool routing done ;)

-Mike

ICI Support wrote:
> 
> The webpage was helpful in some ways, but I think it is a step too advanced
> for me at this time.
I don''t wish to be rude, but if you cannot get what you want from
rc.proxyarp then you need to hire someone with the expertise to do this
for you.
> I guess what I want to know is if the ''plan'' I have is
sound?
It is a sound plan.  Actually, I cannot think of any alternate; that''s
why I did it this way.  So I''d call it the ONLY plan.
 
> I want to put a linux box between my cable modem and my lan. 
It''ll go from
> the cablemodem into eth0 then from eth1 to the LAN.
The only difference between what you describe and rc.proxyarp is that
the interfaces are switched.  You might like to know that rc.proxyarp
runs on an AMD K6 233 CPU with 192Mb RAM, 6Gb (IIRC) HD and 3 NICs; the
point is that it does not take a lot of "horsepower".  eth0 connects
to
a 5 port switch, and the 4 computers with external IPs connect to that
switch.
 
> I just want to do complete passthrough (with packets intact) if possible
> from one side to the other.  This machine doesn''t have to do NAT
or
> anything.
That''s exactly what rc.proxyarp accomplishes.
--
gypsy
 
> Then I want to throttle the bandwidth through that box intelligently (I
kind
> of know how to do that from the Advanced Routing Howto.  I''ve done
it on the
> other three servers to throttle the bandwidth down).
 
> Once all that is working, I''ll deal with using iptables to
firewall.
> 
> If this plan is sound, I guess I need to know the vague steps involved to
> accomplish the routing.  The throttling I can figure out myself.
> 
> -Michael
> 
> > -----Original Message-----
> > From: gypsy [mailto:gypsy@iswest.com]
> > Sent: Monday, June 20, 2005 10:09 PM
> > To: lartc@mailman.ds9a.nl
> > Cc: ICI Support
> > Subject: Re: [LARTC] Help? Router/Bandwidth throttle needed.
> >
> > ICI Support wrote:
> > >
> > > I hope this list is still active.  I''m an experienced
Linux Sysadmin,
> > but I
> > > haven''t done much in the way of routing.  Due to a
decision made by my
> > > higherups, I need to jam a computer between my ISP and my LAN to
do
> > > bandwidth throttling.
> > >
> > > My current setup:
> > >
> > > 1 Crappy Cable Modem (7Mb/768Kb connection) with a static IP.
> > > 4 servers (all have static, routable IPs) - One of which is
running
> > NAT/DHCP
> > > (2 NICs) for 6 workstations (All using DHCP)
> > >
> > > I have 1 spare routable IP.
> > >
> > > My current setup has the cable modem plugged directly into my
LAN.  But
> > it
> > > ends up this modem doesn''t do intelligent
traffic/bandwidth management.
> > I
> > > need a way to throttle and poll usage through the modem such that
> > neither
> > > uploads nor downloads will ever use the full capacity (and thus
dominate
> > the
> > > router).
> > >
> > > Currently, the best I can do, is to throttle some of those 4
servers,
> > but
> > > that doesn''t help me with the workstations.   So I
figured I''d need to
> > jam a
> > > machine between the modem and the LAN to work as a
''gatekeeper'' to
> > control
> > > the bidirectional traffic (and I could use it for a firewall
too).
> > >
> > > Can someone help me on how I need to set this up?  I have a spare
box I
> > > could easily put two NICs into, if needed, to make it the
gatekeeper.
> > >
> > > -----
> > >
> > > Michael Yacht
> > > CTO
> > > Ideal Conditions, Inc.
> > > 5329 Beeler Street, #2
> > > Pittsburgh, PA 15217
> > > v: 412-325-1375
> >
> > Michael,
> >
> > I have exactly your setup except that I have 2 internet connections
and
> > (at long last) neither is crappy.
> >
> > http://yesican.chsoft.biz/lartc
> >
> > The top part of that describes my setup.  Stop when you get to
"Now we
> > have 2 providers".
> >
> > Short description:  eth0 points to the internal servers which have
> > external IPs and I shape the traffic going out on that interface. 
eth1
> > points to the internet and I shape the traffic going out on that.  In
> > order to answer for the 4 internal machines, proxyarp is employed. 
Each
> > of the 4 internal machines has a firewall, but these are a LOT quieter
> > since I put a firewall on this computer <grin>.
> >
> > You may also want to ftp andthatsjazz.org/pub/lartc
> > --
> > gypsy