I''m using 4.4.2 Lite on a gentoo box acting as a router. Shorewall
version 4.4.2.1 is running on the gentoo host.
I have a PPPoE link on DSL advertised as 512kbit/512kbit. It has been
measured at 418kbit/399kbit before queuing beyond my reach occurs.
I have two SIP devices that I want to have precedence over all other
traffic. But I''d like the best possible download/upload if the SIP
devices aren''t in use.
I first set up a simple set of tc* rules following the documentation.
In particular the ingress limit on ppp0 was set to 418kbit.
It worked. In the presence of uploads and downloads there were few
troubles. However for the first 30 seconds or so of a call there''d be
a
lot of dropouts on the incoming side.
I''m pretty sure that the ingress policing filter was indiscriminately
dropping anything it had to get the rate down, some packets being the
incoming RTP stream. After things settled down incoming and outgoing
RTP were fine.
I could have stopped here but I couldn''t leave it alone, I want better.
It seems much more reasonable to apply the ruthless packet dropping only
on the bulk data in the presence of a SIP call. So I set up an IFB device.
I have this now:
---------------------------------------------------------------------------
tcdevices:
1:ppp0 - 399kbit
2:ifb0 - 418kbit classify ppp0
tcclasses:
ppp0 1 170kbit full 1
tos=0x68/0xfc,tos=0xb8/0xfc
ppp0 2 40kbit full 2
tcp-ack,tos-minimize-delay
ppp0 3 189kbit full 3 default
2:110 - 170kbit full 1
tos=0x68/0xfc,tos=0xb8/0xfc
2:120 - 40kbit full 2
tcp-ack,tos-minimize-delay
2:130 - 208kbit full 3 default
tcfilters:
2:110 $VOIP_SP1 0.0.0.0/0 udp - 5060
2:110 $VOIP_SP1 0.0.0.0/0 udp - - -
256
2:120 0.0.0.0/0 0.0.0.0/0 icmp
2:120 0.0.0.0/0 0.0.0.0/0 udp 53
2:120 0.0.0.0/0 0.0.0.0/0 udp 123
($VOIP_SP1 is the IP address of my Voip provider. All incoming SIP/RTP
packets do not have TOS fields set and I have no other clue how to
reliably match the SIP/RTP packets. The outgoing SIP/RTP packets I send
do have properly set TOS fields.)
tcrules: (for completeness only)
2 0.0.0.0/0 0.0.0.0/0 icmp
2 0.0.0.0/0 0.0.0.0/0 udp 53
2 0.0.0.0/0 0.0.0.0/0 udp 123
2 0.0.0.0/0 0.0.0.0/0 tcp:syn
2 $FW 0.0.0.0/0 icmp
2 $FW 0.0.0.0/0 udp 53
2 $FW 0.0.0.0/0 udp 123
2 $FW 0.0.0.0/0 tcp:syn
# voip
RESTORE 0.0.0.0/0 0.0.0.0/0 all - - -
0
CONTINUE 0.0.0.0/0 0.0.0.0/0 all - - -
!0
1 0.0.0.0/0 0.0.0.0/0 all - - -
- - - - sip
SAVE 0.0.0.0/0 0.0.0.0/0 all - - -
!0
---------------------------------------------------------------------------
The filters and marking all seem to be working properly. But overall
it doesn''t work. There is a huge incoming queue building up on the far
side of the DSL line that''s revealed by pinging. Ping times are up to
3.2 seconds during a download. SIP is impossible.
I fought with it for a while and gave up. I fought and gave up several
times.
In the generated "firewall" script there was a command:
run_tc qdisc add dev ifb0 parent 2:130 handle 130: sfq quantum $quantum limit
127 perturb 10
I ruthlessly patched it to:
run_tc qdisc add dev ifb0 parent 2:130 handle 130: sfq quantum $quantum limit 3
perturb 10
The intent was to get the 2:130 class dropping bulk packets as quickly as
possible.
Now it seems to be working exactly as I intended. SIP works with no
obvious dropouts or glitching and downloads/uploads speed up and slow
down rapidly in the absence/presence of SIP calls. Class 2:130 (and
2:130 only) is dropping packets by the boatload and data rates are
maximal. It *looks* perfect. Clearly I''m missing something. :-)
The behaviour is what I intended but I''m not sure if it''s what
I really
"want". ie. It wouldn''t surprise me that there''s
some undesirable
behaviour hidden in there somewhere that I haven''t found yet.
I have no idea if it''s optimal ("3" was a mostly random guess
because
each 1500b packet is 30ms at this incoming rate). Ping times are sub
100ms, presumably (but not tested) a "2" would make ping times sub
60ms.
I have no idea if there isn''t a much better way to achieve the same
effect.
Comments or hints would be appreciated.
If by some fluke this method really is the best way to drop bulk packets
only, then I might suggest some way of setting the "limit" on the tc
qdisc sfq command directly from shorewall.
thank you,
- paul
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
Paul R. Bernard wrote:> I fought with it for a while and gave up. I fought and gave up several > times. > > In the generated "firewall" script there was a command: > > run_tc qdisc add dev ifb0 parent 2:130 handle 130: sfq quantum $quantum limit 127 perturb 10 > > I ruthlessly patched it to: > > run_tc qdisc add dev ifb0 parent 2:130 handle 130: sfq quantum $quantum limit 3 perturb 10 > > The intent was to get the 2:130 class dropping bulk packets as quickly as > possible. > > Now it seems to be working exactly as I intended. SIP works with no > obvious dropouts or glitching and downloads/uploads speed up and slow > down rapidly in the absence/presence of SIP calls. Class 2:130 (and > 2:130 only) is dropping packets by the boatload and data rates are > maximal. It *looks* perfect. Clearly I''m missing something. :-)Hard to argue with a solution that works :-)> > The behaviour is what I intended but I''m not sure if it''s what I really > "want". ie. It wouldn''t surprise me that there''s some undesirable > behaviour hidden in there somewhere that I haven''t found yet.The only down side that I can see is that a sudden burst of traffic through a limited class can cause packet loss. But from what you have observed, that doesn''t seem to be a noticeable issue.> > I have no idea if it''s optimal ("3" was a mostly random guess because > each 1500b packet is 30ms at this incoming rate). Ping times are sub > 100ms, presumably (but not tested) a "2" would make ping times sub 60ms. > > I have no idea if there isn''t a much better way to achieve the same > effect. > > Comments or hints would be appreciated. > > If by some fluke this method really is the best way to drop bulk packets > only, then I might suggest some way of setting the "limit" on the tc > qdisc sfq command directly from shorewall.This is the only way that I can see to accomplish what you want. Attached, please find a patch that implements a ''limit='' OPTION in /etc/shorewall/tcclasses. Valid values are 3-127 with 127 continuing to be the default. This patch (along with updates to the documentation) will be in 4.4.3. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Tom Eastep wrote:> This is the only way that I can see to accomplish what you want. > Attached, please find a patch that implements a ''limit='' OPTION in > /etc/shorewall/tcclasses. Valid values are 3-127 with 127 continuing to > be the default. > > This patch (along with updates to the documentation) will be in 4.4.3.The maximum value for limit in 4.4.3 will 128 rather than 127. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Tom Eastep <teastep@shorewall.net> writes:> Paul R. Bernard wrote:>> The behaviour is what I intended but I''m not sure if it''s what I really >> "want". ie. It wouldn''t surprise me that there''s some undesirable >> behaviour hidden in there somewhere that I haven''t found yet. > > The only down side that I can see is that a sudden burst of traffic > through a limited class can cause packet loss. But from what you have > observed, that doesn''t seem to be a noticeable issue.Not so far anyways. It hasn''t been running for very long. It seems to me that if such issues appear where the loss can''t be tolerated I can create another class to contain the protocol in question with the appropriate priority and keep it away from the "lossy" bulk class. I did note that at the very beginning of the download (in the test noted below) there is an incoming bandwidth spike. I expect it could impact SIP but I''ve not tested this with "limit=3" yet. It did observe something like it when I was using the brute force ingress policing, but it wasn''t a huge issue and could be easily ignored.>> only, then I might suggest some way of setting the "limit" on the tc >> qdisc sfq command directly from shorewall. > > This is the only way that I can see to accomplish what you want. > Attached, please find a patch that implements a ''limit='' OPTION in > /etc/shorewall/tcclasses. Valid values are 3-127 with 127 continuing to > be the default. > > This patch (along with updates to the documentation) will be in 4.4.3.I''ve patched, installed and tested my version. It works. For posterity I''ve attached a "shorewall-lite show tc" after performing a short test. I started a single http download, after it stabilised at 49KBytes/s (according to wget) I started a "ping -c 50" to the upstream router, and then made a 50 second phonecall. The transfer reacted smoothly and very quickly (according to wget) when the call started and ended. "limit=3" might not be optimal, but it''s good enough for now. The ping results throughout the call were: 50 packets transmitted, 50 received, 0% packet loss, time 49072ms rtt min/avg/max/mdev = 14.621/70.633/179.614/38.314 ms For comparison purposes a similar ping on an idle link: 50 packets transmitted, 50 received, 0% packet loss, time 49049ms rtt min/avg/max/mdev = 14.798/23.548/202.591/27.162 ms Thank you, I think I can stop for a while. :-) - paul ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Paul R. Bernard wrote:>I did note that at the very beginning of the download (in the test noted >below) there is an incoming bandwidth spike. I expect it could impact >SIP but I''ve not tested this with "limit=3" yet. It did observe >something like it when I was using the brute force ingress policing, but >it wasn''t a huge issue and could be easily ignored.My guess is that it''s a feature of the upstream rate control by your ISP - we get something similar on the dedicated fibre link at work. In our case, at the other end of a 100m fibre link is a Cisco router configured with bandwidth control to limit traffic to the 6m we pay for. When it first went in, I noticed that starting from below max rate, a transfer could peak at something above 6m for a few seconds and would then settle down. I expect they use a burst setting (like the burst setting in htb) as that will give the appearance to a user sat behind a web browser of a faster line. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Simon Hobson <linux@thehobsons.co.uk> writes:> Paul R. Bernard wrote: > >>I did note that at the very beginning of the download (in the test noted >>below) there is an incoming bandwidth spike....>>I did observe >>something like it when I was using the brute force ingress policing, but >>it wasn''t a huge issue and could be easily ignored.> My guess is that it''s a feature of the upstream rate control by your > ISP - we get something similar on the dedicated fibre link at work.You could be right. (The dedicated fibre at work is similar to yours, it bursts at 10mbits for the first 1MByte of the connection then drops to 3mbits. It was advertised as such. I have a very "consumer" unfriendly DSL modem in that it tells me almost everything about how the link is configured and how it''s performing (can''t see the queueing directly though). It says 512kbit/512kbit for the current connection and also says the maximum possible rate in and out (which varies greatly from 1mbit incoming to 3.2mbit from day to day). I assumed that the 512kbit/512kbit was a hard limit, but maybe the thing can burst a bit. I don''t know enough about ATM/DSL/ISP/phone companies. I haven''t made precise and careful measurements either. I''ve measured an absolute maximum incoming transfer rate of 440kbit with a 3-4 second queue on upstream. It could also be that the "bursting" I''m seeing is all on my side between my 418kbit setting and the maximum I can get. I don''t know how to control it if it is under my control. It hasn''t been much of a problem so far. - paul ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Hello, I am trying to setup a Talkswitch telephone system with off site ip extensions. The sales talk is that by using ip you can have a ''remote'' telephone extension that can be configured to answer the main main office telephone lines. Just like in the office were you can have one extension on a pbx answer any of the incoming calls to the main numbers. I have been able to make it work sporadically but only for the initial call. Then the ip extension phone fails to register with the Talkswitch system in the office and the extension goes down. I know it connects because you can reboot the extension an it is loaded with main office system configuration. This is my first foray into VOIP and I cannot determine if the shorewall firewall may be the stumbling block. Their support gave be the usual, open ports so and so and maybe use the DMZ blah blah. I tried everything from isolation in the DMZ to port forwarding. BTW we have our own 27 ip range. Any guidance would be appreciated. regards, michaelp ------------------------------------------------------------------------------
Michael Parker wrote:>I am trying to setup a Talkswitch telephone system with off site ip >extensions. The sales talk is that by using ip you can have a ''remote'' >telephone extension that can be configured to answer the main >main office telephone lines. Just like in the office >were you can have one extension on a pbx answer any of the incoming >calls to the main numbers. > >I have been able to make it work sporadically but only for the initial >call. Then the ip extension phone fails to register with the >Talkswitch system in the office and the extension goes down. I know it >connects because you can reboot the extension an it is loaded with >main office system configuration. > >This is my first foray into VOIP and I cannot determine if the >shorewall firewall may be the stumbling block. > >Their support gave be the usual, open ports so and so and maybe use >the DMZ blah blah. I tried everything from isolation in the DMZ to >port forwarding. BTW we have our own 27 ip range.Assuming this is "yet another SIP based PBX" ... I''d put it on it''s own external IP - it makes things a lot easier to deal with, SIP is really not nice with NAT. You then need to allow the ports through to the device according to the support docs you have. I''d guess port 5060 UDP & TCP for SIP, and whatever UDP ports it uses for the RTP streams. For phones in the office, out a second NIC in the PBX on the internal network, and have the internal devices talk to it using the internal address - that eliminates NAT between the office phones and the PBX. Now, for your external devices, this is where the fun starts ! While debugging, start off with a phone on one of your public IPs - and when you''ve got that working fully, move it to a ''foreign'' network. Since this will almost certainly be behind NAt then you have a number of options : 1) Let the NAT gateway fixup the SIP packets - many have SIP helpers these days, though my experience is such that I turn them off. 2) Let the phone use STUN to discover it''s network. An external STUN server can help the phone work out what the outside IP address is, and how the gateway is doing the NAT. In many cases this will be sufficient, but if you have a Zyxel gateway doing NAT - forget it, they scramble the port numbers in such a way that this will never work. When I''ve spoken to their technical people, they seem to have the attitude of "never mind if it breaks things, it''s secure" ! 3) If you have a fixed IP and control of the NAT gateway, most devices have the facility for you to tell them what their external IP is, and they''ll use this in their SIP packets. Again, with most NAT gateways that preserve port numbers if possible, this works fine - if you have multiple devices behind a NAT gateway, then make sure they all use different SIP and RTP ports. This also doesn''t work behind a Zyxel gateway. 4) Use a SIP proxy. I don''t know of any, so if anyone know of a FOSS package I''d be keen to hear about it. I know Gradwell (who''s services we resell at work) have hardware based proxies that deal with this. Essentially, the proxy gets the SIP and RTP packets from the device, ignores the address/ports in the SIP messages and uses the actual address/port the packets come from. It then talks to the SIP registrar in place of the end device. Since it ignores the address/port info from the client, this even works with a Zyxel gateway ! Note that Linux has included a SIP NAT helper module for some time, you may want to turn this off (prevent the module loading). -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. ------------------------------------------------------------------------------
Hello Simon, Tuesday, April 27, 2010, 3:04:01 AM, you wrote:> Michael Parker wrote:>>I am trying to setup a Talkswitch telephone system with off site ip >>extensions. The sales talk is that by using ip you can have a ''remote'' >>telephone extension that can be configured to answer the main >>main office telephone lines. Just like in the office >>were you can have one extension on a pbx answer any of the incoming >>calls to the main numbers.Thanks for the leads. I''ll get everyone updated. -- Best regards, Michael Parker mailto:metp@yrts.ca ------------------------------------------------------------------------------