Nick
2012-Jan-09 19:27 UTC
Shorewall gateway - routing issue with dual wan (looking to report possible bug ?)
Hi, I have dual wans setup very similar to the dual wan guide. I used two modems in bridged mode, and PPPOE to authenticate on a Gentoo box. Simply put the issue is at times my ISP hands out the same gateway address on both connections. When this happens shorewall fails to start. Most of the time the two ISP gateways are 203.33.255.118 and 203.33.255.161 though randomly when pppoe restarts I will be assigned the same gateway to both connections. I do have two static IP''s that are assigned from my ISP via DHCP. I can simulate this by editing the providers file like so: NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY isp1 1 512 main eth1 10.100.11.10 track,balance eth0 isp2 2 256 main eth2 10.100.11.10 track,balance eth0 this is the same error that occurs when I am using the pppoe setup: RTNETLINK answers: No such process ERROR: Command "ip -4 route replace default scope global table 254 nexthop via 10.100.11.10 dev eth1 weight 1 nexthop via 10.100.11.10 dev eth2 weight 1" Failed This has driven me mad for 6 months now, and I hope someone can provide a solution other than putting the modems into NAT with permanent different gateways. My knowledge of routing is just enough to get me into trouble. I am happy to provide any further information, I run Gentoo and have updated, patched and rebuilt kernels over the last six months to attempt to sort this. I suspect that Its pretty rare that anyone would have (or want) two identical gateways working on their network on different interfaces, but I hope that at least a simple test could be added to shorewall to prevent this occurring to others. Thanks for reading, Nick. ------------------------------------------------------------------------------ Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don''t need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
Tom Eastep
2012-Jan-09 23:07 UTC
Re: Shorewall gateway - routing issue with dual wan (looking to report possible bug ?)
On Tue, 2012-01-10 at 06:27 +1100, Nick wrote:> I have dual wans setup very similar to the dual wan guide. I used two > modems in bridged mode, and PPPOE to authenticate on a Gentoo box. > > > > Simply put the issue is at times my ISP hands out the same gateway > address on both connections. When this happens shorewall fails to > start. > > > > Most of the time the two ISP gateways are 203.33.255.118 and > 203.33.255.161 though randomly when pppoe restarts I will be > assigned the same gateway to both connections. I do have two static > IP’s that are assigned from my ISP via DHCP.> I can simulate this by editing the providers file like so: > > > > NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY > isp1 1 512 main eth1 10.100.11.10 track,balance eth0 > isp2 2 256 main eth2 10.100.11.10 track,balance eth0> this is the same error that occurs when I am using the pppoe setup: > > RTNETLINK answers: No such process > > ERROR: Command "ip -4 route replace default scope global table 254 > nexthop via 10.100.11.10 dev eth1 weight 1 nexthop via 10.100.11.10 > dev eth2 weight 1" Failed > > This has driven me mad for 6 months now, and I hope someone can > provide a solution other than putting the modems into NAT with > permanent different gateways. My knowledge of routing is just enough > to get me into trouble. > > I am happy to provide any further information, I run Gentoo and have > updated, patched and rebuilt kernels over the last six months to > attempt to sort this. > > I suspect that Its pretty rare that anyone would have (or want) two > identical gateways working on their network on different interfaces, > but I hope that at least a simple test could be added to shorewall to > prevent this occurring to others.With PPPOE, you shouldn''t be specifying the gateway address on your PPP devices. If you just leave the GATEWAY column empty ("-"), this problem shouldn''t occur. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don''t need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
Nick
2012-Jan-10 03:51 UTC
Re: Shorewall gateway - routing issue with dual wan (looking to report possible bug ?)
-----Original Message----- From: Tom Eastep [mailto:teastep@shorewall.net] Sent: Tuesday, 10 January 2012 10:07 AM To: Shorewall Users Subject: Re: [Shorewall-users] Shorewall gateway - routing issue with dual wan (looking to report possible bug ?) On Tue, 2012-01-10 at 06:27 +1100, Nick wrote:> I have dual wans setup very similar to the dual wan guide. I used two > modems in bridged mode, and PPPOE to authenticate on a Gentoo box. > > > > Simply put the issue is at times my ISP hands out the same gateway > address on both connections. When this happens shorewall fails to > start. > > > > Most of the time the two ISP gateways are 203.33.255.118 and > 203.33.255.161 though randomly when pppoe restarts I will be > assigned the same gateway to both connections. I do have two static > IP’s that are assigned from my ISP via DHCP.> I can simulate this by editing the providers file like so: > > > > NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY > isp1 1 512 main eth1 10.100.11.10 track,balance eth0 > isp2 2 256 main eth2 10.100.11.10 track,balance eth0> this is the same error that occurs when I am using the pppoe setup: > > RTNETLINK answers: No such process > > ERROR: Command "ip -4 route replace default scope global table 254 > nexthop via 10.100.11.10 dev eth1 weight 1 nexthop via 10.100.11.10 > dev eth2 weight 1" Failed > > This has driven me mad for 6 months now, and I hope someone can > provide a solution other than putting the modems into NAT with > permanent different gateways. My knowledge of routing is just enough > to get me into trouble. > > I am happy to provide any further information, I run Gentoo and have > updated, patched and rebuilt kernels over the last six months to > attempt to sort this. > > I suspect that Its pretty rare that anyone would have (or want) two > identical gateways working on their network on different interfaces, > but I hope that at least a simple test could be added to shorewall to > prevent this occurring to others.With PPPOE, you shouldn't be specifying the gateway address on your PPP devices. If you just leave the GATEWAY column empty ("-"), this problem shouldn't occur. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ Cheers Tom, Originally I had shorewall set to detect the gateway. This setup was crashing randomly when pppoe restarted the connection with this kind of error: ERROR: Command "ip -4 route replace default scope global table 254 nexthop via 203.33.255.161 dev eth1 weight 1 nexthop via 203.33.255.161 dev eth2 weight 1" Failed Though from memory the error had the DNS name of the gateway. I have stopped this error by putting the modems into NAT mode so they are authenticating using pppoe and give shorewall a static unique gateway. I can reproduce the error by setting the gateways to the same address. I don’t have the ability to control the gateway assigned to me on the PPPOE session and occasionally my ISP will assign me the same gateway via DHCP over the PPPOE connection. Since this setup is reasonably difficult to reproduce I have demonstrated an easy method to reproduce the issue by editing the providers file. I am simply doing this to assist in improving shorewall as I do have a work around. :) Considerable time and effort has gone into researching this issue with nothing found on the internet to suggest that Shorewall is incapable of operating two ISP if they happen to use the same gateway. Hope this helps make it clearer, Nick. ------------------------------------------------------------------------------ Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Tom Eastep
2012-Jan-10 04:41 UTC
Re: Shorewall gateway - routing issue with dual wan (looking to report possible bug ?)
On Jan 9, 2012, at 7:51 PM, Nick wrote:> > Originally I had shorewall set to detect the gateway. This setup was crashing randomly when pppoe restarted the connection with this kind of error: > ERROR: Command "ip -4 route replace default scope global table 254 > nexthop via 203.33.255.161 dev eth1 weight 1 nexthop via 203.33.255.161 > dev eth2 weight 1" FailedDid you see the word ''detect'' in my post?> > I can reproduce the error by setting the gateways to the same address.Which is a configuration that will never work. Neither Shorewall nor the Linux IP stack will handle that.> > I don’t have the ability to control the gateway assigned to me on the PPPOE session and occasionally my ISP will assign me the same gateway via DHCP over the PPPOE connection > > Since this setup is reasonably difficult to reproduce I have demonstrated an easy method to reproduce the issue by editing the providers file. > > I am simply doing this to assist in improving shorewall as I do have a work around. :) > Considerable time and effort has gone into researching this issue with nothing found on the internet to suggest that Shorewall is incapable of operating two ISP if they happen to use the same gateway.I repeat: Place the single character ''-'' in the GATEWAY column for PPOE providers. I am convinced that you will not see any problems with the modems in bridging mode. -Tom Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev
Tom Eastep
2012-Jan-10 04:51 UTC
Re: Shorewall gateway - routing issue with dual wan (looking to report possible bug ?)
On Jan 9, 2012, at 8:41 PM, Tom Eastep wrote:> > On Jan 9, 2012, at 7:51 PM, Nick wrote: >> >> Originally I had shorewall set to detect the gateway. This setup was crashing randomly when pppoe restarted the connection with this kind of error: >> ERROR: Command "ip -4 route replace default scope global table 254 >> nexthop via 203.33.255.161 dev eth1 weight 1 nexthop via 203.33.255.161 >> dev eth2 weight 1" Failed > > Did you see the word ''detect'' in my post? > >> >> I can reproduce the error by setting the gateways to the same address. > > Which is a configuration that will never work. Neither Shorewall nor the Linux IP stack will handle that. > >> >> I don’t have the ability to control the gateway assigned to me on the PPPOE session and occasionally my ISP will assign me the same gateway via DHCP over the PPPOE connection >> >> Since this setup is reasonably difficult to reproduce I have demonstrated an easy method to reproduce the issue by editing the providers file. >> >> I am simply doing this to assist in improving shorewall as I do have a work around. :) >> Considerable time and effort has gone into researching this issue with nothing found on the internet to suggest that Shorewall is incapable of operating two ISP if they happen to use the same gateway. > > > I repeat: Place the single character ''-'' in the GATEWAY column for PPOE providers. I am convinced that you will not see any problems with the modems in bridging mode.Sorry; I don''t mean to be offensive and I appreciate the fact that you are trying to help. But point-to-point interfaces don''t require a gateway because there is only one host at the end of the point-to-point connection. So anything sent through a PTP interface will end up at the ''gateway''. And specifying a gateway explicitly or asking Shorewall to detect the gateway just messes things up. Regards, -Tom Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev
Tom Eastep
2012-Jan-10 15:51 UTC
Re: Shorewall gateway - routing issue with dual wan (looking to report possible bug ?)
On Mon, 2012-01-09 at 20:51 -0800, Tom Eastep wrote:> > On Jan 9, 2012, at 7:51 PM, Nick wrote: > >> I can reproduce the error by setting the gateways to the same address. > > > > Which is a configuration that will never work. Neither Shorewall nor the Linux IP stack will handle that.I should quantify that. Balancing using a multi-hop default route will not work in that case. Over the past couple of weeks, I have been working on an alternative for balancing that does not involve multi-hop routes. It rather uses the ''Statistic Match'' feature in iptables/Netfilter that allows a rule to match randomly with a specified probability. I have been running it here at shorewall.net for the last few days and it seems to work well. It will be available in the next 4.5.0 Beta and will provide relief to users with two WAN Ethernet interfaces that happen to have the same default gateway. Here is my providers file: #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY ComcastB 1 - - eth1 70.90.191.126 loose,balance ComcastC 2 - - eth0 detect loose,fallback I have PROVIDER_OFFSET=16 and PROVIDER_BITS=2 which means that the ''provider mask'' is 0x30000, ComcastB''s mark is 0x10000 and ComcastC''s mark is 0x20000. I also have TRACK_PROVIDERS=Yes. Here are the relevant entries in my tcrules file: ... 0X10000/0x30000 eth2 - ; test=0/0x30000, probability=0.66666667 0x20000/0x30000 eth2 - ; test=0/0x30000 0X10000/0x30000 fw - ; test=0/0x30000, probability=0.66666667 0x20000/0x30000 fw - ; test=0/0x30000 The first two distribute connections from the local LAN (eth2) between the two providers with a 2:1 advantage to ComcastB. The second two perform the same distribution for connections originating on the firewall itself (Note: $FW = ''fw'' in my configuration). I include 0/0x30000 in the TEST column because earlier rules may have already marked to packet based on other criteria. I hope to be able to make this easier to configure before 4.5.0 final; we''ll see. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev
Nick
2012-Jan-31 08:57 UTC
Re: Shorewall gateway - routing issue with dual wan (looking to report possible bug ?)
Hey, Its taken a while to reproduce, I put the modems back into bridged mode and eventually got this: acfxlinux storage # route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 203.33.255.118 0.0.0.0 255.255.255.255 UH 0 0 0 ppp0 203.33.255.118 0.0.0.0 255.255.255.255 UH 0 0 0 ppp1 10.100.13.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 10.100.12.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 10.1.10.0 192.168.1.30 255.255.255.0 UG 2 0 0 eth0 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 127.0.0.0 127.0.0.1 255.0.0.0 UG 0 0 0 lo 0.0.0.0 203.33.255.118 0.0.0.0 UG 4005 0 0 ppp0 0.0.0.0 203.33.255.118 0.0.0.0 UG 4006 0 0 ppp1 The good news is after removing the ''detect'' from the providers file its still all working fine even with identical gateways! Many thanks for helping me with this Tom :) -----Original Message----- From: Tom Eastep [mailto:teastep@shorewall.net] Sent: Wednesday, 11 January 2012 2:51 AM To: Shorewall Users Subject: Re: [Shorewall-users] Shorewall gateway - routing issue with dual wan (looking to report possible bug ?) On Mon, 2012-01-09 at 20:51 -0800, Tom Eastep wrote:> > On Jan 9, 2012, at 7:51 PM, Nick wrote: > >> I can reproduce the error by setting the gateways to the same address. > > > > Which is a configuration that will never work. Neither Shorewall nor the Linux IP stack will handle that.I should quantify that. Balancing using a multi-hop default route will not work in that case. Over the past couple of weeks, I have been working on an alternative for balancing that does not involve multi-hop routes. It rather uses the ''Statistic Match'' feature in iptables/Netfilter that allows a rule to match randomly with a specified probability. I have been running it here at shorewall.net for the last few days and it seems to work well. It will be available in the next 4.5.0 Beta and will provide relief to users with two WAN Ethernet interfaces that happen to have the same default gateway. Here is my providers file: #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY ComcastB 1 - - eth1 70.90.191.126 loose,balance ComcastC 2 - - eth0 detect loose,fallback I have PROVIDER_OFFSET=16 and PROVIDER_BITS=2 which means that the ''provider mask'' is 0x30000, ComcastB''s mark is 0x10000 and ComcastC''s mark is 0x20000. I also have TRACK_PROVIDERS=Yes. Here are the relevant entries in my tcrules file: ... 0X10000/0x30000 eth2 - ; test=0/0x30000, probability=0.66666667 0x20000/0x30000 eth2 - ; test=0/0x30000 0X10000/0x30000 fw - ; test=0/0x30000, probability=0.66666667 0x20000/0x30000 fw - ; test=0/0x30000 The first two distribute connections from the local LAN (eth2) between the two providers with a 2:1 advantage to ComcastB. The second two perform the same distribution for connections originating on the firewall itself (Note: $FW = ''fw'' in my configuration). I include 0/0x30000 in the TEST column because earlier rules may have already marked to packet based on other criteria. I hope to be able to make this easier to configure before 4.5.0 final; we''ll see. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d