LARTC - Aug 2007 - subdivide 64 kbit bandwidth 32kbit for WWW and 32 Kbit for mail

Hi all,

I''ve got a BOX  running CentOS 4.5. It acts as a firewall + router.

I have installed both iptables and iproute2.

I has 3 network cards.

eth0 is connected to Internet (is has an internet ip. pls assume its ip is
1.2.3.4/29). it is a 256 Kbit link.
eth1 is DMZ. its ip is 192.168.100.254
eth2 is LAN. Its ip is 192.168.101.254

I have alreday shaped traffic to 64 Kbit on eth1 for downloadig (i.e DMZ
zone ).

This is the sript that does the job. It works ine.

#traffic shaping on eth1 (Downloading)

INTERFAZ_DMZ=eth1
FULLBANDWIDTH=256
BANDWIDTH4DMZ=64

tc qdisc del root dev $INTERFAZ_DMZ

tc qdisc add dev $INTERFAZ_DMZ root handle 1: htb r2q 4
tc class add dev $INTERFAZ_DMZ parent 1: classid 1:2 htb rate
"$FULLBANDWIDTH"Kbit
tc class add dev $INTERFAZ_DMZ parent 1: classid 1:5 htb rate
"$BANDWIDTH4DMZ"Kbit
tc qdisc add dev $INTERFAZ_DMZ parent 1:5 handle 5: sfq perturb 10
tc filter add dev $INTERFAZ_DMZ parent 1: protocol ip prio 1 u32 match ip
dst 192.168.100.0/24 classid 1:5


It has allocated 64 Kbit for downloading for the ip range of
192.168.100.0/24. (DMZ ZONE)

Rememmber, this is a SNATed firewall.

Now, What I nedd is to subdivide this 64 kbit bandwidth *32kbit for WWW and
32 Kbit for mail**.

Can I subdivide in that way ? If divided , What will happen to other
services such as ICMP, SSH, ACK etc ?

*Then, How can I achieve this task?
*
I modfied the the above script . This is what it looks like after editing.



*#traffic shaping on eth1 (Downloading)

INTERFAZ_DMZ=eth1
FULLBANDWIDTH=256
BANDWIDTH4DMZ=64

tc qdisc del root dev $INTERFAZ_DMZ

tc qdisc add dev $INTERFAZ_DMZ root handle 1: htb r2q 4
tc class add dev $INTERFAZ_DMZ parent 1: classid 1:2 htb rate
"$FULLBANDWIDTH"Kbit
tc class add dev $INTERFAZ_DMZ parent 1: classid 1:5 htb rate
"$BANDWIDTH4DMZ"Kbit

**tc class add dev $INTERFAZ_DMZ parent 1:5 classid 1:10 htb rate 32kbit
tc class add dev $INTERFAZ_DMZ parent 1:5 classid 1:11 htb rate 32Kbit

tc qdisc add dev $INTERFAZ_DMZ parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $INTERFAZ_DMZ parent 1:11 handle 11: sfq perturb 10

#192.168.100.3 is the BOX acts as a mail server and a proxyserver.
tc filter add dev $INTERFAZ_DMZ parent 1: protocol ip prio 1 u32 match ip
dst 192.168.100.0/24 classid 1:10
tc filter add dev $INTERFAZ_DMZ parent 1: protocol ip prio 1 u32 match ip
dst 192.168.100.0/24 match ip dport 25  classid 1:11



Pls let me know if it is Okay?  or any better way to rewrite it?

EXPECT YOUR COMMENTS.





****

-- 
Thank you
Indunil Jayasooriya


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Indunil Jayasooriya wrote:
> Hi all,
> 
> I''ve got a BOX  running CentOS 4.5. It acts as a firewall +
router.
> 
> I have installed both iptables and iproute2.
> 
> I has 3 network cards.
> 
> eth0 is connected to Internet (is has an internet ip. pls assume its ip is
> 1.2.3.4/29). it is a 256 Kbit link.
> eth1 is DMZ. its ip is 192.168.100.254
> eth2 is LAN. Its ip is 192.168.101.254
> 
> I have alreday shaped traffic to 64 Kbit on eth1 for downloadig (i.e DMZ
> zone ).
> 
> This is the sript that does the job. It works ine.
> 
> #traffic shaping on eth1 (Downloading)
> 
> INTERFAZ_DMZ=eth1
> FULLBANDWIDTH=256
> BANDWIDTH4DMZ=64
> 
> tc qdisc del root dev $INTERFAZ_DMZ
> 
> tc qdisc add dev $INTERFAZ_DMZ root handle 1: htb r2q 4
> tc class add dev $INTERFAZ_DMZ parent 1: classid 1:2 htb rate
> "$FULLBANDWIDTH"Kbit
> tc class add dev $INTERFAZ_DMZ parent 1: classid 1:5 htb rate
> "$BANDWIDTH4DMZ"Kbit
> tc qdisc add dev $INTERFAZ_DMZ parent 1:5 handle 5: sfq perturb 10
> tc filter add dev $INTERFAZ_DMZ parent 1: protocol ip prio 1 u32 match ip
> dst 192.168.100.0/24 classid 1:5
> 
> 
> It has allocated 64 Kbit for downloading for the ip range of
> 192.168.100.0/24. (DMZ ZONE)
> 
> Rememmber, this is a SNATed firewall.
> 
> Now, What I nedd is to subdivide this 64 kbit bandwidth *32kbit for WWW and
> 32 Kbit for mail**.
Do you want to share 64kbit so if there is no mail then www can have all 
64kbit?
> 
> Can I subdivide in that way ? If divided , What will happen to other
> services such as ICMP, SSH, ACK etc ?
You need to make your rules to allow for these as well - depending on 
what other traffic hits the server it may be best to give everything 
other than big tcp www/mail packets priority.
> 
> *Then, How can I achieve this task?
> *
> I modfied the the above script . This is what it looks like after editing.
> 
> 
> 
> *#traffic shaping on eth1 (Downloading)
IT can be hard to shape properly from the wrong end of a slow wan - but 
your rates here are low so it should be OK.
> 
> INTERFAZ_DMZ=eth1
> FULLBANDWIDTH=256
> BANDWIDTH4DMZ=64
> 
> tc qdisc del root dev $INTERFAZ_DMZ
> 
> tc qdisc add dev $INTERFAZ_DMZ root handle 1: htb r2q 4
> tc class add dev $INTERFAZ_DMZ parent 1: classid 1:2 htb rate
> "$FULLBANDWIDTH"Kbit
> tc class add dev $INTERFAZ_DMZ parent 1: classid 1:5 htb rate
> "$BANDWIDTH4DMZ"Kbit
> 
> **tc class add dev $INTERFAZ_DMZ parent 1:5 classid 1:10 htb rate 32kbit
> tc class add dev $INTERFAZ_DMZ parent 1:5 classid 1:11 htb rate 32Kbit
> 
> tc qdisc add dev $INTERFAZ_DMZ parent 1:10 handle 10: sfq perturb 10
> tc qdisc add dev $INTERFAZ_DMZ parent 1:11 handle 11: sfq perturb 10
> 
> #192.168.100.3 is the BOX acts as a mail server and a proxyserver.
> tc filter add dev $INTERFAZ_DMZ parent 1: protocol ip prio 1 u32 match ip
> dst 192.168.100.0/24 classid 1:10
> tc filter add dev $INTERFAZ_DMZ parent 1: protocol ip prio 1 u32 match ip
> dst 192.168.100.0/24 match ip dport 25  classid 1:11
> 
If these go in in order of entry (they usually do if prio is the same , 
but not always!) then nothing will reach 1:11.
> 
> 
> Pls let me know if it is Okay?  or any better way to rewrite it?
It depends what you want and on your setup. Do you have traffic from LAN 
to the proxy/mail - do you really need to shape that or not?

Do you have traffic from the internet to LAN as well - do you need to 
shape that - maybe sharing bandwidth with DMZ.

> 
> EXPECT YOUR COMMENTS.
> 
> 
> 
> 
> 
> ****
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

>
>
> >
> > Now, What I need is to subdivide this 64 kbit bandwidth *32kbit for
WWW
> and
> > 32 Kbit for mail**.
>
> Do you want to share 64kbit so if there is no mail then www can have all
> 64kbit?

 When there is no mail, WWW should take all 64 kbit and also when there is
no WWW, mail should take all 64 kbit.

remember. This is only for downloading. NOT for UPLAODING as this is a
SNATed firewll.

pls see below for SNATed rules.


#SNAT from LAN1 192.168.101.0/24
iptables -t nat -A POSTROUTING -p tcp -o eth0 -s 192.168.101.0/24 -m
multiport --dports 20,21,69,80,443 -j SNAT --to-source 203.143.26.130
iptables -t nat -A POSTROUTING -p udp -o eth0 -s 192.168.101.0/24 --dport
1024: -j SNAT --to-source 203.143.26.130
iptables -t nat -A POSTROUTING -p tcp -o eth0 -s 192.168.101.0/24 --dport
1024: -j SNAT --to-source 203.143.26.130
iptables -t nat -A POSTROUTING -p udp -o eth0 -s 192.168.101.0/24 --dport 53
-j SNAT --to-source 203.143.26.130
iptables -t nat -A POSTROUTING -p icmp -o eth0 -s 192.168.101.0/24 -j SNAT
--to-source 203.143.26.130


#SNAT from DMZ ip address of 192.168.100.3 (mail and proxy server)

iptables -t nat -A POSTROUTING -p tcp -o eth0 -s 192.168.100.3 -m multiport
--dports 21,22,25,80,443 -j SNAT --to-source 203.143.26.130
iptables -t nat -A POSTROUTING -p udp -o eth0 -s 192.168.100.3 --dport 53 -j
SNAT --to-source 203.143.26.130
iptables -t nat -A POSTROUTING -p icmp -o eth0 -s 192.168.100.3 -j SNAT
--to-source 203.143.26.130

>
> > Can I subdivide in that way ? If divided , What will happen to other
> > services such as ICMP, SSH, ACK etc ?
>
> You need to make your rules to allow for these as well - depending on
> what other traffic hits the server it may be best to give everything
> other than big tcp www/mail packets priority.

Server only acts as a mail server and a proxy server. in addition to that, I
ssh to that server from LAN. from that server too, I ssh to some servers.
And also, I make ping to that server rom LAN. Again, from that server I make
ping to other servers. That''s it.

How can I make such rules?
>
> > *Then, How can I achieve this task?
> > *
> > I modfied the the above script . This is what it looks like after
> editing.
> >
> >
> >
> > *#traffic shaping on eth1 (Downloading)
>
> IT can be hard to shape properly from the wrong end of a slow wan - but
> your rates here are low so it should be OK.

64 kbit is the allocated bandwidth for DMZ. If needed, I can make it to 128
kbit.

>
> > INTERFAZ_DMZ=eth1
> > FULLBANDWIDTH=256
> > BANDWIDTH4DMZ=64
> >
> > tc qdisc del root dev $INTERFAZ_DMZ
> >
> > tc qdisc add dev $INTERFAZ_DMZ root handle 1: htb r2q 4
> > tc class add dev $INTERFAZ_DMZ parent 1: classid 1:2 htb rate
> > "$FULLBANDWIDTH"Kbit
> > tc class add dev $INTERFAZ_DMZ parent 1: classid 1:5 htb rate
> > "$BANDWIDTH4DMZ"Kbit
> >
> > **tc class add dev $INTERFAZ_DMZ parent 1:5 classid 1:10 htb rate
32kbit
> > tc class add dev $INTERFAZ_DMZ parent 1:5 classid 1:11 htb rate 32Kbit
> >
> > tc qdisc add dev $INTERFAZ_DMZ parent 1:10 handle 10: sfq perturb 10
> > tc qdisc add dev $INTERFAZ_DMZ parent 1:11 handle 11: sfq perturb 10
> >
> > #192.168.100.3 is the BOX acts as a mail server and a proxyserver.
> > tc filter add dev $INTERFAZ_DMZ parent 1: protocol ip prio 1 u32 match
> ip
> > dst 192.168.100.0/24 classid 1:10
> > tc filter add dev $INTERFAZ_DMZ parent 1: protocol ip prio 1 u32 match
> ip
> > dst 192.168.100.0/24 match ip dport 25  classid 1:11
> >
>
> If these go in in order of entry (they usually do if prio is the same ,
> but not always!) then nothing will reach 1:11.

Then,  What will have to do?  How can I write  the script properly?
>
> >
> > Pls let me know if it is Okay?  or any better way to rewrite it?
>
> It depends what you want and on your setup. Do you have traffic from LAN
> to the proxy/mail - do you really need to shape that or not?

my DMZ  is 192.168.100.0/24
my LAN is 192.168.101.0/24

LAN users access my DMZ proxy and mail server (its ip is 192.168.100.3) as I
have DNATed as bellow.

#DNAT from LAN1 to ip 192.168.100.3
iptables -t nat -A PREROUTING -p tcp -i eth2 -d 192.168.101.254 --dport 25
-j DNAT --to-destination 192.168.100.3:25
iptables -t nat -A PREROUTING -p tcp -i eth2 -d 192.168.101.254 --dport 3128
-j DNAT--to-destination 192.168.100.3:3128

Tha''s it. NO NEED to shape this.

Do you have traffic from the internet to LAN as well - do you need
to
> shape that - maybe sharing bandwidth with DMZ.

LAN users actualy browse  internet and send and recieve mails  via  DMZ
proxy server and Mail server. No other  traffic.

Thanks for you comments. Hope to hear from you.
>
> > EXPECT YOUR COMMENTS.
> >
> >
> >
> >
> >
> > ****
> >
> >
> >
> >
------------------------------------------------------------------------
> >
> > _______________________________________________
> > LARTC mailing list
> > LARTC@mailman.ds9a.nl
> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>

-- 
Thank you
Indunil Jayasooriya


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Indunil Jayasooriya wrote:
> Server only acts as a mail server and a proxy server. in addition to that,
I
> ssh to that server from LAN. from that server too, I ssh to some servers.
> And also, I make ping to that server rom LAN. Again, from that server I
make
> ping to other servers. That''s it.
> 
> LAN users actualy browse  internet and send and recieve mails  via  DMZ
> proxy server and Mail server. No other  traffic.
> 
> Thanks for you comments. Hope to hear from you.
I think the easiest way would be to make some netfilter rules to mark 
traffic coming in on eth0 and going out on eth1, and then make filters 
to match the marks.

You could do it with just tc filters, but it''s easier to write using 
iptables.

Something like (just an example - you may want to be more specific with 
the rules/add more and debug them!)

iptables -t mangle -A FORWARD -i eth0 -o eth1 -p tcp -m length --length 
128: -j MARK --set-mark 3

So tcp from internet to dmz bigger than 128 gets mark 3

iptables -t mangle -A FORWARD -i eth0 -o eth1 -p tcp --dport 25 -m mark 
--mark 3 -j MARK --set-mark 2

big mail packets remarked to 2 so I am just assuming other big tcp is www

iptables -t mangle -A FORWARD -i eth0 -o eth1 -m mark --mark 0 -j MARK 
--set-mark 1

Anything else unmarked form internet to dmz gets mark 1


tc qdisc del dev eth1 root &>/dev/null
tc qdisc add dev eth1 root handle 1:0 htb

tc class add dev eth1 parent 1:0 classid 1:1 htb rate 64kbit quantum 1514

tc class add dev eth1 parent 1:1 classid 1:10 htb rate 44kbit ceil 
64kbit quantum 1514 prio 0
tc qdisc add dev eth1 parent 1:10 handle 10: bfifo limit 64k
tc filter add dev eth1 parent 1:0 prio 1 protocol ip handle 1 fw flowid 1:10

tc class add dev eth1 parent 1:1 classid 1:20 htb rate 10kbit ceil 
64kbit quantum 1514 prio 1
tc qdisc add dev eth1 parent 1:20 handle 20: sfq perturb 10 limit 10
tc filter add dev eth1 parent 1:0 prio 2 protocol ip handle 2 fw flowid 1:20

tc class add dev eth1 parent 1:1 classid 1:30 htb rate 10kbit ceil 
64kbit quantum 1514 prio 1
tc qdisc add dev eth1 parent 1:30 handle 30: sfq perturb 10 limit 10
tc filter add dev eth1 parent 1:0 prio 3 protocol ip handle 3 fw flowid 1:30

This is just an untested example - I don''t even run a mail
server/proxy.

You could, I suppose use sfq instead of bfifo for the small/non tcp 
class to help if you get flooded with syns or something. I gave it a 
higher rate even though it shouldn''t have much traffic because htb
seems
to give better latency that way.

The 10 packet limit on sfqs will cause drops, but should help keep 
latency low, but you may want to test and increase it.

If you don''t already shape on egress I would at least do something like
the above so that sending big mail doesn''t lag out the connection. Just
making big tcp second class with sfq will stop DNS getting delayed by a 
backlogged link.

Andy.