Hi,
first of all, thanks that there exists Shorewall! I really, really love that
project (since many years).
I have set up an ISP router gateway with advanced routing and TC stuff using
shorewall. There are 2 things that I do not know how to solve directly in
shorewall, so I have used a hand made TC script and some rules in
/etc/shorewall/started.
My question is, if there exists a way do do it directly with shorewall.
If a clinet connect with PPPoE, accel-ppp (the PPPoE server) call
/etc/ppp/ip-up/somescript and sets TC rules for each new pppX interface. It
reads its up and down values from /var/run/radattr.pppX, which is written by the
RADIUS server on connect.
This is my ip-up script:
--------------------------------------------------------------
PPP_IFACE="$1"
PPP_TTY="$2"
PPP_SPEED="$3"
PPP_LOCAL="$4"
PPP_REMOTE="$5"
PPP_IPPARAM="$6"
# Lock this resource
for wait_for_lock in $(seq 1 60); do
if [ -e /tmp/lock-$PPP_IFACE ]; then
sleep 1
else
touch /tmp/lock-$PPP_IFACE
break
fi
done
IP=/bin/ip
TC=/sbin/tc
BANDUP=`grep RP-Upstream-Speed-Limit /var/run/radattr.${PPP_IFACE} | cut -d
" " -f 2`
BANDDOWN=`grep RP-Downstream-Speed-Limit /var/run/radattr.${PPP_IFACE} | cut -d
" " -f 2`
# deltaweb-services
MAXDOWN=81920
MAXUP=${MAXDOWN}
echo -n " Clearing tc root, ingress... "
${TC} qdisc del dev ${PPP_IFACE} root 2> /dev/null > /dev/null
${TC} qdisc del dev ${PPP_IFACE} ingress 2> /dev/null > /dev/null
echo "done."
echo -n " Adding tc classes... "
# add HFSC root qdisc
${TC} qdisc add dev ${PPP_IFACE} root handle 1: hfsc default 121
# add main rate limit class
${TC} class add dev ${PPP_IFACE} parent 1:0 classid 1:1 hfsc \
sc rate ${MAXDOWN}kibit \
ul rate ${MAXDOWN}kibit
# interactive
${TC} class add dev ${PPP_IFACE} parent 1:1 classid 1:11 hfsc \
sc umax 1500b dmax 30ms rate $[${BANDDOWN}/20]kibit \
ul rate $[${BANDDOWN}/20]kibit
${TC} class add dev ${PPP_IFACE} parent 1:1 classid 1:12 hfsc \
sc rate $[${BANDDOWN}*3/4]kibit \
ul rate ${BANDDOWN}kibit
# ultraFast
${TC} class add dev ${PPP_IFACE} parent 1:1 classid 1:13 hfsc \
sc rate $[${MAXDOWN}/2]kibit \
ul rate ${MAXDOWN}kibit
# default
${TC} class add dev ${PPP_IFACE} parent 1:12 classid 1:121 hfsc \
sc umax 1500b dmax 53ms rate $[${BANDDOWN}/2]kibit \
ul rate ${BANDDOWN}kibit
# large downloads 50Mb - 1000Mb
${TC} class add dev ${PPP_IFACE} parent 1:12 classid 1:122 hfsc \
sc rate $[${BANDDOWN}/2]kibit \
ul rate $[${BANDDOWN}/2]kibit
# large downloads 1000Mb+
${TC} class add dev ${PPP_IFACE} parent 1:12 classid 1:123 hfsc \
sc rate $[${BANDDOWN}/10]kibit \
ul rate $[${BANDDOWN}/5]kibit
# P2P
${TC} class add dev ${PPP_IFACE} parent 1:12 classid 1:124 hfsc \
sc rate 64kibit \
ul rate 64kibit
echo "done."
echo -n " Adding tc qdiscs... "
${TC} qdisc add dev ${PPP_IFACE} parent 1:11 sfq perturb 10
${TC} qdisc add dev ${PPP_IFACE} parent 1:121 handle 121: sfq perturb 10
${TC} qdisc add dev ${PPP_IFACE} parent 1:122 sfq perturb 10
${TC} qdisc add dev ${PPP_IFACE} parent 1:123 sfq perturb 10
${TC} qdisc add dev ${PPP_IFACE} parent 1:124 pfifo
${TC} qdisc add dev ${PPP_IFACE} parent 1:13 sfq perturb 10
echo "done."
echo -n " Adding tc filters... "
${TC} filter add dev ${PPP_IFACE} parent 1:0 protocol ip prio 10 u32 \
match ip tos 0x10 0xff \
flowid 1:13
# marked interactive traffic
${TC} filter add dev ${PPP_IFACE} protocol all parent 1:0 prio 20 handle 0x1 fw
classid 1:11
# ultraFast
${TC} filter add dev ${PPP_IFACE} protocol all parent 1:0 prio 20 handle 0xc fw
classid 1:13
# large downloads 50Mb - 1000Mb
${TC} filter add dev ${PPP_IFACE} protocol all parent 1:0 prio 40 handle 0x2 fw
classid 1:122
# large downloads 1000Mb+
${TC} filter add dev ${PPP_IFACE} protocol all parent 1:0 prio 50 handle 0x3 fw
classid 1:123
# P2P
${TC} filter add dev ${PPP_IFACE} protocol all parent 1:0 prio 60 handle 0x4 fw
classid 1:124
echo "done."
echo -n " Adding tc ingress, filters... "
${TC} qdisc add dev ${PPP_IFACE} handle ffff: ingress
# deltaweb server - 1. subnet
${TC} filter add dev ${PPP_IFACE} parent ffff: protocol all prio 10 u32 \
match ip dst 193.239.107.16/28 \
police rate $[${MAXUP}]kibit \
burst 80kb drop \
flowid :1
# deltaweb server - 2. subnet
${TC} filter add dev ${PPP_IFACE} parent ffff: protocol all prio 10 u32 \
match ip dst 193.239.107.48/28 \
police rate $[${MAXUP}]kibit \
burst 80kb drop \
flowid :1
# RNS server
${TC} filter add dev ${PPP_IFACE} parent ffff: protocol all prio 10 u32 \
match ip dst 193.239.107.32/28 \
police rate $[${MAXUP}]kibit \
burst 80kb drop \
flowid :1
${TC} filter add dev ${PPP_IFACE} parent ffff: protocol all prio 10 u32 \
match ip src 0.0.0.0/0 \
police rate $[${BANDUP}]kibit \
burst 80kb drop \
flowid :1
echo "done."
# Remove (stale) lock file
rm -f /tmp/lock-$PPP_IFACE
--------------------------------------------------------------
Shorewall is doing the MARKing in tcrules:
--------------------------------------------------------------
##
## PPPoE:
##
COMMENT Copy connmark to packet mark
RESTORE/0x00FF:F \
- - all
COMMENT SIP
CONTINUE:F - - all - - - 0x1
COMMENT P2P
CONTINUE:F - - all - - - 0x4
COMMENT Services deltaweb/RNS
CONTINUE:F - - all - - - 0xC
COMMENT Sipgate
0x1:F $DWNET $SIPGATE1 udp
COMMENT Sipgate
0x1:F $DWNET $SIPGATE2 udp
COMMENT Sipgate
0x1:F $DWNET $SIPGATE3 udp
COMMENT Easybell
0x1:F $DWNET $EASYBELL udp
COMMENT
0x1:F - - udp 5060:5076
0x1:F - - udp - 5060:5076
0x1:F - - udp 5004:5020
0x1:F - - udp - 5004:5020
SAVE/0x00FF:F - - udp - - - 0x1
CONTINUE:F - - udp - - - 0x1
0xC:F $KVM1 $DWNET all
0xC:F $KVM2 $DWNET all
0xC:F $RNS $DWNET all
SAVE/0x00FF:F - - all - - - 0xC
CONTINUE:F - - all - - - 0xC
0x2:F - - tcp - - - - - - 52428800:1048576000
0x2:F - - udp - !$UDP_EXCEPTIONS \
- - - - 52428800:1048576000
0x3:F - - tcp - - - - - - 1048576000:
0x3:F - - udp - !$UDP_EXCEPTIONS \
- - - - 1048576000:
0x4:F - - ipp2p:all \
edk,dc,gnu,kazaa,bit,apple,winmx,soul,ares
SAVE/0x00FF:F - - all - - - 0x4
CONTINUE:F - - all - - - 0x4
--------------------------------------------------------------
On shorewall restart, started is called:
--------------------------------------------------------------
#!/bin/bash
###############################################################################
# DO NOT EDIT THIS FILE!! UNDER SALTSTACK CONTROL!! #
###############################################################################
TC=/sbin/tc
for ppp in $(ip -4 add list | grep "global ppp" | awk '{ print $7;
}')
do
echo "${ppp}:"
/etc/ppp/ip-up.d/99-rns-limits ${ppp}
done
echo -n "Adding filters to bond1.108, ifb0..."
${TC} filter del dev bond1.108 protocol all parent 1:0 prio 5 handle 0x1 fw
classid 1:110 >/dev/null 2>&1
${TC} filter del dev ifb0 protocol all parent 2:0 prio 5 handle 0x1 fw classid
2:110 >/dev/null 2>&1
${TC} filter del dev bond1.108 protocol all parent 1:0 prio 5 handle 0x4 fw
classid 1:150 >/dev/null 2>&1
${TC} filter del dev ifb0 protocol all parent 2:0 prio 5 handle 0x4 fw classid
2:150 >/dev/null 2>&1
${TC} filter add dev bond1.108 protocol all parent 1:0 prio 5 handle 0x1 fw
classid 1:110
${TC} filter add dev ifb0 protocol all parent 2:0 prio 5 handle 0x1 fw classid
2:110
${TC} filter add dev bond1.108 protocol all parent 1:0 prio 5 handle 0x4 fw
classid 1:150
${TC} filter add dev ifb0 protocol all parent 2:0 prio 5 handle 0x4 fw classid
2:150
echo " done"
return 0
--------------------------------------------------------------
So now the first question is, can I somehow call shorewall from inside the ip-up
script and set up all TC directly in shorewall, as I already have done for the
internet connection? So I could replace my script and let shorewall do the job.
The other question is already visible in my started script. It’s the TC filter
rules.
I have set up an ifb0 interface, which mirrors the outgoing line (tcclasses):
--------------------------------------------------------------
#NUMBER: IN-BANDWITH OUT-BANDWIDTH OPTIONS REDIRECTED
#INTERFACE INTERFACES
1:bond1.108 - 10mbit classify
2:ifb0 - 10mbit - bond1.108
3:bond1.200 50mbit 10mbit
4:bond1.201 25mbit 5mbit
--------------------------------------------------------------
So with shorewall I have set most of my rules in tcfilters. But I could not find
a way to set filters based on packet marks, so I added the lines above in
started, which of course is not so great.
I could not find any good reason on the net, why setting such rules on ifb0 woul
not make sense. Both rules use egress and on www.linuxfoundation.org I also
found examples like the one in my started script. So basically settings such
filters should be possivle, should it?
0x1 mark in the example above is traffic that is SIP and has its on class. 0x5
are all the ipp2p things, which we do not really want and so we shape it down:
--------------------------------------------------------------
#INTERFACE:CLASS MARK RATE: CEIL PRIORITY OPTIONS
# DMAX:UMAX
# bond1.108
1:110 - 2mbit 2mbit 1 tos=0x68/0xfc,tos=0xb8/0xfc
1:120 - 512kbit 2mbit 2 tcp-ack,tos-minimize-delay
1:130 - 5mbit 6mbit 3
1:140 - 2mbit 6mbit 4 default
1:150 - 128kbit 128kbit 5 pfifo
# ifb0
2:110 - 2mbit 2mbit 1 tos=0x68/0xfc,tos=0xb8/0xfc
2:120 - 512kbit 2mbit 2 tcp-ack,tos-minimize-delay
2:130 - 5mbit 6mbit 3
2:140 - 2mbit 6mbit 4 default
2:150 - 128kbit 128kbit 5 pfifo
3:110 0x20 10mbit 10mbit 1 default
4:110 0x20 5mbit 5mbit 1 default
--------------------------------------------------------------
I hope my questions are okay. I really do not want to waste anybodys time. It is
just that I am not sure, if I already found the optimal way of doing things. And
it already has some complexity. At least for me :)
Ah, just forgot: this is all on Debian Wheezy, shorewall version:
lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 7.4 (wheezy)
Release: 7.4
Codename: wheezy
shorewall version: 4.5.5.3
I also put all shorewall stuff together and attached it to this mail.
Kind regards
-Christian Rößner
--
[*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
-Christian Rößner
--
[*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
------------------------------------------------------------------------------