Hello shorewall-users, The Problem I have a box with 3 network interfaces: * eth0 -> ISP 1 * eth1 -> ISP 2 * eth2 -> local network This box runs asterisk, and I would like to balance SIP and RTP traffic on both ISPs. Solution Rationale I have configured both ISPs on the providers file, hence shorewall will (route) balance packets on both interfaces. Since I do not know in advance the interface the packets will flow out, I would like to let shorewall do its magic with the first packet, and put a connection mark on the packet when it comes back from one of the ISPs, and then use this connection mark to assure following packets keep flowing to the same interface. Implementation Attempt Attached follows my shorewall dump. Here I briefly discuss what I consider to be the main aspect of the solution. Except from tcrules: ### interface selection rules ### route packages thru selected interfaces RESTORE - - all - - - 0 CONTINUE - - all - - - !0 # the first sip packet will leave by any of the connections # when we get a sip response, mark the connection # based on the interface it comes in. 0x100 eth0 - udp 5060 0x200 eth1 - udp 5060 # save connection mark SAVE - - udp 5060 This (tries to) implement the solution rationale. However, no packets are marked by the high-mark rules (by shorewall show mangle). Can someone help me spot what I am missing, or a better approach? Thanks in advance, Pedro ------------------------------------------------------------------------------ Xperia(TM) PLAY It''s a major breakthrough. An authentic gaming smartphone on the nation''s most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev
On 4/6/11 7:02 AM, Pedro Bulach Gapski wrote:> Hello shorewall-users, > > The Problem > > I have a box with 3 network interfaces: > * eth0 -> ISP 1 > * eth1 -> ISP 2 > * eth2 -> local network > This box runs asterisk, and I would like to balance SIP and RTP > traffic on both ISPs. > > Solution Rationale > > I have configured both ISPs on the providers file, hence shorewall > will (route) balance packets on both interfaces. Since I do not know > in advance the interface the packets will flow out, I would like to > let shorewall do its magic with the first packet, and put a connection > mark on the packet when it comes back from one of the ISPs, and then > use this connection mark to assure following packets keep flowing to > the same interface. > > Implementation Attempt > > Attached follows my shorewall dump. Here I briefly discuss what I > consider to be the main aspect of the solution. > Except from tcrules: > ### interface selection rules > ### route packages thru selected interfaces > RESTORE - - all - - - 0 > CONTINUE - - all - - - !0 > # the first sip packet will leave by any of the connections > # when we get a sip response, mark the connection > # based on the interface it comes in. > 0x100 eth0 - udp 5060 > 0x200 eth1 - udp 5060 > # save connection mark > SAVE - - udp 5060 > > This (tries to) implement the solution rationale. However, no packets > are marked by the high-mark rules (by shorewall show mangle). > > Can someone help me spot what I am missing, or a better approach?Simply set the ''track'' and ''balance'' options on both providers and Shorewall will do this for you. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Xperia(TM) PLAY It''s a major breakthrough. An authentic gaming smartphone on the nation''s most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev
Hello Tom, On Wed, Apr 6, 2011 at 11:53 AM, Tom Eastep <teastep@shorewall.net> wrote:> On 4/6/11 7:02 AM, Pedro Bulach Gapski wrote: >> Hello shorewall-users, >> >> The Problem >> >> I have a box with 3 network interfaces: >> * eth0 -> ISP 1 >> * eth1 -> ISP 2 >> * eth2 -> local network >> This box runs asterisk, and I would like to balance SIP and RTP >> traffic on both ISPs. >> >> Solution Rationale >> >> I have configured both ISPs on the providers file, hence shorewall >> will (route) balance packets on both interfaces. Since I do not know >> in advance the interface the packets will flow out, I would like to >> let shorewall do its magic with the first packet, and put a connection >> mark on the packet when it comes back from one of the ISPs, and then >> use this connection mark to assure following packets keep flowing to >> the same interface. >> >> Implementation Attempt >> >> Attached follows my shorewall dump. Here I briefly discuss what I >> consider to be the main aspect of the solution. >> Except from tcrules: >> ### interface selection rules >> ### route packages thru selected interfaces >> RESTORE - - all - - - 0 >> CONTINUE - - all - - - !0 >> # the first sip packet will leave by any of the connections >> # when we get a sip response, mark the connection >> # based on the interface it comes in. >> 0x100 eth0 - udp 5060 >> 0x200 eth1 - udp 5060 >> # save connection mark >> SAVE - - udp 5060 >> >> This (tries to) implement the solution rationale. However, no packets >> are marked by the high-mark rules (by shorewall show mangle). >> >> Can someone help me spot what I am missing, or a better approach? > > Simply set the 'track' and 'balance' options on both providers and > Shorewall will do this for you.I already have. I must be missing something obvious, but I see no traffic flowing thru eth1. Will the route cache keep *all* traffic from localhost to the same outside host on the same interface for good?> > -Tom > -- > Tom Eastep \ When I die, I want to go like my Grandfather who > Shoreline, \ died peacefully in his sleep. Not screaming like > Washington, USA \ all of the passengers in his car > http://shorewall.net \________________________________________________ > > > ------------------------------------------------------------------------------ > Xperia(TM) PLAY > It's a major breakthrough. An authentic gaming > smartphone on the nation's most reliable network. > And it wants your games. > http://p.sf.net/sfu/verizon-sfdev > _______________________________________________ > Shorewall-users mailing list > Shorewall-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/shorewall-users > >------------------------------------------------------------------------------ Xperia(TM) PLAY It's a major breakthrough. An authentic gaming smartphone on the nation's most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Tom Eastep wrote:>Simply set the ''track'' and ''balance'' options on both providers and >Shorewall will do this for you.But won''t fix his other problem - that the IP address in his SIP packets will be wrong for one or other of the ISPs. I doubt if the SIP ALG in the kernel would deal with this either. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. ------------------------------------------------------------------------------ Xperia(TM) PLAY It''s a major breakthrough. An authentic gaming smartphone on the nation''s most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev
On 4/6/11 8:33 AM, Pedro Bulach Gapski wrote:>> Simply set the ''track'' and ''balance'' options on both providers and >> Shorewall will do this for you. > > I already have. I must be missing something obvious, but I see no > traffic flowing thru eth1. Will the route cache keep *all* traffic > from localhost to the same outside host on the same interface for > good?Yes. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Xperia(TM) PLAY It''s a major breakthrough. An authentic gaming smartphone on the nation''s most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev
On Wed, Apr 6, 2011 at 1:11 PM, Simon Hobson <linux@thehobsons.co.uk> wrote:> Tom Eastep wrote: > >>Simply set the ''track'' and ''balance'' options on both providers and >>Shorewall will do this for you. > > But won''t fix his other problem - that the IP address in his SIP > packets will be wrong for one or other of the ISPs. I doubt if the > SIP ALG in the kernel would deal with this either. >This is indeed a major problem. In this case I have configured no NAT on the box itself and I am counting on the ISP modems to work out the NAT translations of its traffic. On isolated tests using route_rules to select the interface, it works with both ISPs separately. My problem now is really in achieving a nice load balance, but since the endpoints are always the same this has been a challenge and I am running out of ideas :-( Pedro> -- > Simon Hobson > > Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed > author Gladys Hobson. Novels - poetry - short stories - ideal as > Christmas stocking fillers. Some available as e-books. > > ------------------------------------------------------------------------------ > Xperia(TM) PLAY > It''s a major breakthrough. An authentic gaming > smartphone on the nation''s most reliable network. > And it wants your games. > http://p.sf.net/sfu/verizon-sfdev > _______________________________________________ > Shorewall-users mailing list > Shorewall-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/shorewall-users >------------------------------------------------------------------------------ Xperia(TM) PLAY It''s a major breakthrough. An authentic gaming smartphone on the nation''s most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev
Pedro Bulach Gapski wrote:> > But won''t fix his other problem - that the IP address in his SIP >> packets will be wrong for one or other of the ISPs. I doubt if the >> SIP ALG in the kernel would deal with this either. >> > >This is indeed a major problem. In this case I have configured no NAT >on the box itself and I am counting on the ISP modems to work out the >NAT translations of its traffic. On isolated tests using route_rules >to select the interface, it works with both ISPs separately.Sounds like your provider is running a NAT-proxy service - that should work round that problem.>My problem now is really in achieving a nice load balance, but since >the endpoints are always the same this has been a challenge and I am >running out of ideas :-(Have you considered a different approach ? Could you create two sets of trunks, possibly bound to different IPs, and let the exchange pseudo-balance the calls across the trunks ? I''m assuming the (or at least one) reason for trying to load balance is to avoid having n+1 calls on a connection that only supports n, so having a trunk routed via that connection and limited to n calls would deal with that. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. ------------------------------------------------------------------------------ Xperia(TM) PLAY It''s a major breakthrough. An authentic gaming smartphone on the nation''s most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev
On Wed, Apr 6, 2011 at 2:39 PM, Simon Hobson <linux@thehobsons.co.uk> wrote:> Pedro Bulach Gapski wrote: > >> > But won't fix his other problem - that the IP address in his SIP >>> packets will be wrong for one or other of the ISPs. I doubt if the >>> SIP ALG in the kernel would deal with this either. >>> >> >>This is indeed a major problem. In this case I have configured no NAT >>on the box itself and I am counting on the ISP modems to work out the >>NAT translations of its traffic. On isolated tests using route_rules >>to select the interface, it works with both ISPs separately. > > Sounds like your provider is running a NAT-proxy service - that > should work round that problem. > >>My problem now is really in achieving a nice load balance, but since >>the endpoints are always the same this has been a challenge and I am >>running out of ideas :-( > > Have you considered a different approach ? Could you create two sets > of trunks, possibly bound to different IPs, and let the exchange > pseudo-balance the calls across the trunks ? I'm assuming the (or at > least one) reason for trying to load balance is to avoid having n+1 > calls on a connection that only supports n, so having a trunk routed > via that connection and limited to n calls would deal with that.This may be a nice way to go. I can easily create two trunks and balance the calls between them, and we already have call limits in place. But I still need a way to bound each trunk to a different interface, since the endpoints are always the same. Maybe I can use something akin to iptables -m string --string 'TRUNK1' and iptables -m string 'TRUNK2' to place tcrules-like marks on the packets based on the trunk. Is there a way to access iptables -m string functionality from inside shorewall in a tcrules-compatible way?> > -- > Simon Hobson > > Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed > author Gladys Hobson. Novels - poetry - short stories - ideal as > Christmas stocking fillers. Some available as e-books. > > ------------------------------------------------------------------------------ > Xperia(TM) PLAY > It's a major breakthrough. An authentic gaming > smartphone on the nation's most reliable network. > And it wants your games. > http://p.sf.net/sfu/verizon-sfdev > _______________________________________________ > Shorewall-users mailing list > Shorewall-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/shorewall-users >------------------------------------------------------------------------------ Xperia(TM) PLAY It's a major breakthrough. An authentic gaming smartphone on the nation's most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Pedro Bulach Gapski wrote:>This may be a nice way to go. I can easily create two trunks and >balance the calls between them, and we already have call limits in >place. > >But I still need a way to bound each trunk to a different interface, >since the endpoints are always the same.*If* you can bind to two IPs and use different addresses for the trunks, then you could use policy based routing on source address. Don''t know how to do it though ! -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. ------------------------------------------------------------------------------ Xperia(TM) PLAY It''s a major breakthrough. An authentic gaming smartphone on the nation''s most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev
This post summarizes what has been done for the reference of others. The initial goal was to load balance SIP traffic flowing from a single box to two different internet connections on two different interfaces. The traffic generated from asterisk version 1.4 (debian lenny). The attempted solution was as follows: 1. create two trunks for SIP calls 2. split peers between these two trunks 3. bind each trunk to a different interface 4. register asterisk on both trunks AFAIK, 1, 2 and 3 were successful, but 4 failed because of SIP NAT problems and asterisk limitations (to my best knowledge). Steps 1 and 2 were done in the asterisk configuration. Step 3 was done with shorewall. A standard configuration was used as the base: a. interfaces listed in the providers file; b. traffic shaping in tcrules. Additionally, an extension script was used to bind each SIP trunk to a different interface, using iptables -m string extensions. I ended using the ''started'' shorewall entry-point with the following configuration: run_iptables -t mangle -A OUTPUT -d 200.219.209.250 -p udp --dport 5060 -m string --string "<sip:59799@200.219.209.250>" --algo bm -j MARK --set-mark 0x100 run_iptables -t mangle -A OUTPUT -d 200.219.209.250 -p udp --dport 5060 -m string --string "<sip:28736@200.219.209.250>" --algo bm -j MARK --set-mark 0x200 run_iptables -t mangle -A OUTPUT -d 200.219.209.250 -p udp --dport 5060 -j CONNMARK --save-mark I call this ''one ugly hack'', but it worked :-P Traffic was correctly assigned to the determined interface. Shorewall rocks :-) Step 4 failed, it seems, because asterisk binds to all interfaces but always uses the address of the first interface in the Via header, or the configured unique external ip. Without a configured external ip, traffic bound to the second interface contains a Via header with the address of the first interface, and is not replied to. With a configured extenal ip, the Via header always uses this address, which is appropriate for one of the interfaces but not the other. I believe this could be made to work by patching asterisk to allow for different external ip for each trunk. Comments and suggestions are welcome. Pedro ------------------------------------------------------------------------------ Xperia(TM) PLAY It''s a major breakthrough. An authentic gaming smartphone on the nation''s most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev
On 04/07/2011 12:25 PM, Pedro Bulach Gapski wrote:> Step 3 was done with shorewall. A standard configuration was used as the base: > a. interfaces listed in the providers file; > b. traffic shaping in tcrules. > > Additionally, an extension script was used to bind each SIP trunk to a > different interface, using iptables -m string extensions. I ended > using the ''started'' shorewall entry-point with the following > configuration: > run_iptables -t mangle -A OUTPUT -d 200.219.209.250 -p udp --dport > 5060 -m string --string "<sip:59799@200.219.209.250>" --algo bm -j > MARK --set-mark 0x100 > run_iptables -t mangle -A OUTPUT -d 200.219.209.250 -p udp --dport > 5060 -m string --string "<sip:28736@200.219.209.250>" --algo bm -j > MARK --set-mark 0x200 > run_iptables -t mangle -A OUTPUT -d 200.219.209.250 -p udp --dport > 5060 -j CONNMARK --save-markOne usually doesn''t use run_iptables() in the ''started'' script; that is only appropriate in the ''start'' script. In ''started'', simply use $IPTABLES (e.g., $IPTABLES -t mangle -A OUTPUT ...) -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Xperia(TM) PLAY It''s a major breakthrough. An authentic gaming smartphone on the nation''s most reliable network. And it wants your games. http://p.sf.net/sfu/verizon-sfdev
This turned out to be more complex than I had initially planned ;-) This post quickly summarizes the final working setup. The initial goal was to balance traffic between two outbound interfaces. This was accomplished with the following: * one master asterisk that handles local SIP peers * slave asterisks running on openvz VEs, one for each external outbound interface The master asterisk registers with both VE asterisks, and has one context to forward calls to each asterisk VE. SIP peers are statically split between both contexts. Each VE asterisk acts as a simple proxy, accepting registration from the master and registering with the VoIP provider. The shorewall configuration is based on the openvz+shorewall guidelines. Additionally, it routes outbound traffic from each VE to a specific interface through entries in route_rules. Furthermore, traffic shaping is used to prioritize VoIP traffic. Downsides: completely static and no failover at all! Regards, Pedro On Thu, Apr 7, 2011 at 4:25 PM, Pedro Bulach Gapski <pbgapski@gmail.com> wrote:> This post summarizes what has been done for the reference of others. > > The initial goal was to load balance SIP traffic flowing from a single > box to two different internet connections on two different interfaces. > The traffic generated from asterisk version 1.4 (debian lenny). > > The attempted solution was as follows: > 1. create two trunks for SIP calls > 2. split peers between these two trunks > 3. bind each trunk to a different interface > 4. register asterisk on both trunks > > AFAIK, 1, 2 and 3 were successful, but 4 failed because of SIP NAT > problems and asterisk limitations (to my best knowledge). > > Steps 1 and 2 were done in the asterisk configuration. > > Step 3 was done with shorewall. A standard configuration was used as the base: > a. interfaces listed in the providers file; > b. traffic shaping in tcrules. > > Additionally, an extension script was used to bind each SIP trunk to a > different interface, using iptables -m string extensions. I ended > using the 'started' shorewall entry-point with the following > configuration: > run_iptables -t mangle -A OUTPUT -d 200.219.209.250 -p udp --dport > 5060 -m string --string "<sip:59799@200.219.209.250>" --algo bm -j > MARK --set-mark 0x100 > run_iptables -t mangle -A OUTPUT -d 200.219.209.250 -p udp --dport > 5060 -m string --string "<sip:28736@200.219.209.250>" --algo bm -j > MARK --set-mark 0x200 > run_iptables -t mangle -A OUTPUT -d 200.219.209.250 -p udp --dport > 5060 -j CONNMARK --save-mark > > I call this 'one ugly hack', but it worked :-P Traffic was correctly > assigned to the determined interface. Shorewall rocks :-) > > Step 4 failed, it seems, because asterisk binds to all interfaces but > always uses the address of the first interface in the Via header, or > the configured unique external ip. Without a configured external ip, > traffic bound to the second interface contains a Via header with the > address of the first interface, and is not replied to. With a > configured extenal ip, the Via header always uses this address, which > is appropriate for one of the interfaces but not the other. I believe > this could be made to work by patching asterisk to allow for different > external ip for each trunk. > > Comments and suggestions are welcome. > > Pedro >------------------------------------------------------------------------------ Forrester Wave Report - Recovery time is now measured in hours and minutes not days. Key insights are discussed in the 2010 Forrester Wave Report as part of an in-depth evaluation of disaster recovery service providers. Forrester found the best-in-class provider in terms of services and vision. Read this report now! http://p.sf.net/sfu/ibm-webcastpromo _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users