Hi. I''m wondering if there''s a good way to configure a Linux firewall box to failover to a single backup server, while preserving connection state. This question has been asked before, but the latest reference I can find is from 2004, at which time Linux had no equivalent of OpenBSD''s pfsync, though Harald was said to be working on one. Did anything come of those efforts? Or is there now another alternative? Any examples or advice would be appreciated. Thank you. -- ams
At 2007-07-19 22:32:51 +0530, ams@toroid.org wrote:> > I''m wondering if there''s a good way to configure a Linux firewall box > to failover to a single backup server, while preserving connection > state.Looks like this is it: http://people.netfilter.org/pablo/conntrack-tools/ -- ams
On 07/19/07 12:02, Abhijit Menon-Sen wrote:> I''m wondering if there''s a good way to configure a Linux firewall box > to failover to a single backup server, while preserving connection > state.I''m a bit confused, are you wanting a single Linux firewall / router to have redundant internet connections, or to route traffic to redundant systems behind it and intelligently handle the failure of one or more of said redundant systems? I''m also not sure how conntrackd (comparable to OpenBSD''s pfsync) is coming in to play here. Or is there more than one Linux firewall / router that you are wanting to synchronize? Or are you wanting the connection tracking between the multiple systems behind the Linux firewall / router? I think that all of these are possible to various degrees, though each uses a different method to achieve it.> This question has been asked before, but the latest reference I can > find is from 2004, at which time Linux had no equivalent of OpenBSD''s > pfsync, though Harald was said to be working on one.*nod* Conntrackd is the tool that you want to use to synchronize connection tracking connection meta data between two systems, or the closest thing that Linux presently has (that I''m aware of).> Did anything come of those efforts? Or is there now another > alternative?Yes, conntrackd.> Any examples or advice would be appreciated.Will you please clarify what you are really wanting to do per above and I''ll be more than happy to try to point you in the right direction. Grant. . . .
Hi Grant. At 2007-07-19 16:15:01 -0500, gtaylor@riverviewtech.net wrote:> > I''m a bit confused, are you wanting a single Linux firewall / > router to have redundant internet connections, or to route > traffic to redundant systems behind it and intelligently > handle the failure of one or more of said redundant systems?Neither. I just want a hot standby for a single Linux firewall, such that clients behind it are not affected by a hardware failure on the firewall. If my configuration would allow me to someday promote the backup and run both firewall machines in a load-balancing configuration, so much the better. The following example looks very much like what I want: http://people.netfilter.org/pablo/conntrack-tools/testcase.html (Can anyone comment on whether I should stick with keepalived as described above, or try out ucarp?)> Will you please clarify what you are really wanting to do per > above and I''ll be more than happy to try to point you in the > right direction.Thanks, I''d appreciate any advice you can give me. -- ams
Abhijit Menon-Sen wrote:> Hi Grant. > > At 2007-07-19 16:15:01 -0500, gtaylor@riverviewtech.net wrote: >> I''m a bit confused, are you wanting a single Linux firewall / >> router to have redundant internet connections, or to route >> traffic to redundant systems behind it and intelligently >> handle the failure of one or more of said redundant systems? > > Neither. > > I just want a hot standby for a single Linux firewall, such that clients > behind it are not affected by a hardware failure on the firewall. If my > configuration would allow me to someday promote the backup and run both > firewall machines in a load-balancing configuration, so much the better. > > The following example looks very much like what I want: > > http://people.netfilter.org/pablo/conntrack-tools/testcase.html > > (Can anyone comment on whether I should stick with keepalived as > described above, or try out ucarp?) > >> Will you please clarify what you are really wanting to do per >> above and I''ll be more than happy to try to point you in the >> right direction. > > Thanks, I''d appreciate any advice you can give me. > > -- amsIn case your firewall is a proxy for some service, those connections will fail though - unless you can use a virtual interface with the same IP as the source for such connections. I guess you''ll use vrrp in conjunction for failover. It would make sense to use vrrpd with status tracking of WAN gateway but AFAIK no such feature exists as yet. Mohan
On 07/19/07 20:20, Abhijit Menon-Sen wrote:> I just want a hot standby for a single Linux firewall, such that > clients behind it are not affected by a hardware failure on the > firewall. If my configuration would allow me to someday promote the > backup and run both firewall machines in a load-balancing > configuration, so much the better.Ok, to pull this off I would use a pair of Linux boxen with vrrpd and conntrackd with almost identical configurations on each system (save for host name and management IPs). For the sake of discussion I''ll presume that you are using a cable modem or adsl connection to the internet with a static IP address. PPPoE / DHCP should be possible, but it will get much nastier. You will really want a minimum of two static IP addresses, three if possible, on each side of the firewall. One IP address will be for the routing of traffic and the other two will be for management. The one routing IP address will be the virtual IP that is passed back and forth between the systems and also the IP address that it looks like all your traffic passes through. Use VRRP to make sure that one of the systems will always have the routing IP up and functional.> The following example looks very much like what I want: > > http://people.netfilter.org/pablo/conntrack-tools/testcase.html*nod* This is very much what you are wanting to do. However I''m not sure that the direct connection between the two systems is absolutely required. However having not done this my self I can not say for sure. The last time I looked at this I was going to be trying to make redundant routers for a 1 - 3 Mbps connection and thus was planing on using VLANs to do Router-on-a-stick across two bonded 100 BaseT connections. So, if I needed to I could create a new VLAN for the router to router communications.> (Can anyone comment on whether I should stick with keepalived as > described above, or try out ucarp?)I would use vrrpd over keepalived. As far as ucarp, I''m not familiar with it so I can''t comment. I can say that keepalived is much more complex and can do a lot more than what you need to do. Now if you were running a load balancing cluster of servers behind it, keepalived is the proper answer. However for just making sure that a router is available, vrrpd is much closer to what you need with out overkilling. In fact, I believe vrrpd only needs command line options and no config file where as, last I looked, keepalived has a very complex config file for all that it could do. With regards to "... run both firewall machines in a load-balancing configuration ..." I think that conntrackd will allow you to do this. At the very least, you can have inbound traffic use one firewall as its primary and outbound use the other firewall as its primary where each is a failover for the other. If you want inbound and / or outbound to use both routers at the same time, it gets very trick to have the layer 2 ethernet network know which router to use to send the traffic to. A quick run down of how VRRP (and Cisco''s proprietary HSRP) works for those that may not know or want a refresher. VRRP uses a management IP and a virtual routing (that may or may not be one of the management IPs). The virtual routing IP uses a special MAC address (00-00-5E-00-01-<virtual router ID>). This special MAC address allows all client workstations using the virtual routing IP to be able to cache the MAC address and not have to re-arp for the system that currently has the IP. In the event that the backup router detects that the primary router is down, the backup router will claim to be the virtual IP on the virtual router MAC address, usually GARPing so that switches now see the virtual router MAC address on the new switch port. Thus when clients try to send traffic out through the virtual router, the layer 2 traffic will be to the virtual MAC address which the switches now know to be elsewhere on the functioning router. The net result is a very brief down time while the backup router is detecting that the primary is not functioning and switches over. (Note: This is a very far fetch problem, but is possible.) The only real problem with this scenario is if for some reason both routers are up and functioning, but for some reason they can not see each other, thus they both think the other node is down. In this case, you may have a battle for who is up. If you want to know what to do in this situation read about SONITH (Shoot Other Node In The Head) to make sure that there is only one active node at a time. You can configure VRRP to automatically switch back from the backup to the primary when the primary comes back on line, or let the backup remain the primary until it fails and then the real primary that is acting as the backup will resume its role as the primary.> Thanks, I''d appreciate any advice you can give me.Having a functioning understanding of what is going on will help make this a success. What you are wanting to do is completely possible and should be able to be achieved with out a lot of problems, but it is out side of the scope of any point and click GUI that I have seen for Linux. Good luck and let me know if there is any thing else that I can do to help. Grant. . . .
On 07/19/07 20:32, Mohan Sundaram wrote:> It would make sense to use vrrpd with status tracking of WAN gateway > but AFAIK no such feature exists as yet.If I understand what you are saying, you are referring to an external T-1 router converting from the WAN circuit to the ethernet going in to the two routers right? If this is the case, no matter what is done, the WAN router is a single point of failure and thus can not be avoided. No matter what, the Linux boxen can be configured such that they will try to reach this single point of failure and in the event that they can not do so, they (both) will ultimately return an ICMP "no route to host" error message back to the client. However this is out side the scope of what VRRP is meant to do. Grant. . . .
At 2007-07-20 10:12:01 -0500, gtaylor@riverviewtech.net wrote:> > > I just want a hot standby for a single Linux firewall [...] > > I would use a pair of Linux boxen with vrrpd and conntrackdOK, great. I didn''t know about vrrpd. I''ll check it out.> As far as ucarp, I''m not familiar with it so I can''t comment.If I have the time, I''ll try out ucarp and post a summary of my experiences for the archives.> If you want to know what to do in this situation read about SONITH > (Shoot Other Node In The Head) to make sure that there is only one > active node at a time.("STONITH", for those asking Google.) I have one other question. How does conntrackd interact with traffic shaping? My firewall also uses HTB to impose various bandwidth limits on clients. From what I''ve read so far, I have the impression that the failover may lose some packets that are being delayed in a queue, but existing connections should recover and be esentially unaffected. Can anyone confirm that? -- ams
On 7/20/2007 9:03 PM, Mohan Sundaram wrote:> I think my point was misunderstood. Let us say each of these Linux > boxes are connected to a WAN link each. If the WAN gateway/link of a > box goes down, vrrp must flag itself down or as secondary. Some > similarity to our earlier discussions on redundant gateways. This > feature of object tracking is available in CISCO (their patent) but > only tracks the interface status and not the gateway reachability. > I''d love to have a feature where gateway reachability is tracked.Each firewall / router / gateway having its own independent wan / internet connection makes things a bit different. First of all, each will have its own public IP address for the associated WAN link and as such probably have it''s own NAT configuration. I wonder, what type of WAN connection are these? Could both be hooked up to both gateways? In other words are they ethernet or something that gets bridged to ethernet or are they some sort of legacy WAN link, i.e. T1, Frame Relay, ATM? If you could connect the WAN links to both systems, you can have even more functionality and you would be back to what I was originally thinking except for the fact that instead of one WAN connection, you have two to make each router aware of. At the very least, I think you will need to make each router aware of the other one. This way, if a router''s (primary) WAN link is not usable it can route the traffic over to the other router and have it route the traffic out to the world. Thus each router would have a primary default gateway of the router at the other end of its WAN link *AND* a secondary default gateway of the other router that it is connected to. I also think that you are very close to needing to use some sort of monitoring utility / daemon to check the status of the WAN link *AND* to make the other router aware of the status. This may be easily done with a small daemon to monitor the link and update the local routing table in conjunction with a routing protocol between the two routers to keep each aware of the others routing table. If you have any more information on Cisco''s technology I''d be interested in doing some reading about it if you would care to toss it my way. Grant. . . .
On 7/20/2007 9:18 PM, Mohan Sundaram wrote:> In VRRP based gateway pairs, this is normally done by partitioning > the LAN to use both gateways by different subnets. i.e G1 is primary > for subnet1 with G2 as secondary; G2 is primary for subnet2 with G1 > as secondary. This is done by defining multiple vrrp groups. AFAIK, > no dynamic balancing methods/ features exist.Agreed, this would be the easiest to implement active / active with traffic flowing out through both routers. I was trying to state that outbound for the network would flow through one router while inbound would flow through the other router. This could also be extended to mirror the multiple subnets like you are talking about too, though I think load sharing reasoning would be lost at that point as you have more traffic on both routers. At this point in time it would come down to statistics of network load to see how you wanted to do it. I''m sure there are ways that you can have all traffic spread across both routers in either direction. Though I''m not quite sure how to go about it. I''m going to say that VRRP / Load Balancing / Bridging / Layer 2 Filtering / and other services would be in effect to divide the traffic across both active routers. No matter how you slice it, this is beyond the scope of the OPs question. I was merely stating that things are possible. Grant. . . .
At 2007-07-21 05:59:54 +0530, ams@toroid.org wrote:> > If I have the time, I''ll try out ucarp and post a summary of my > experiences for the archives.Not much to report. I set up ucarp as directed in the README, and it just worked. It was simple and did what I wanted (which was to allow two machines to share a virtual IP with heartbeat/failover). -- ams