Is there a multi-homing FAQ? I''ve not been able to find it. Here''s my system''s configuration: eth0: SDSL at 216.15.108.186 (dnai-net) eth1: ADSL at 67.114.175.138 (sbc-net) eth2: DHCP network at 192.168.1.1 . lo: 127.0.0.1 There is a firewall box acting as the DHCP server and router for the systems on the DHCP net. The Linux system doesn''t run DHCP or masquerading. If a connection is accepted on the IP address of one of these interfaces, I''d like its packets to go out on the same interface. If there is a failure on the path to eth0, connections to eth1 should continue to work, and vice versa. But right now, eth0 has the default route and just about every packet goes out of eth0 regardless of what interface they come in. So, I tried this: ip rule add from 216.15.108.186 table dnai-net ip rule add from 67.114.175.138 table sbc-net ip route add default via 216.15.108.186 dev eth0 table dnai-net ip route add default via 67.114.175.138 dev eth1 table sbc-net This doesn''t work. I get a destination unreachable if I try to follow either route. So, what am I doing wrong? Thanks Bruce Perens
On Sat, Mar 02, 2002 at 03:49:49PM -0800, Bruce Perens wrote:> Is there a multi-homing FAQ? I''ve not been able to find it.No, but it is very high on my agenda. Multihoming and loadbalancing are among the most popular subjects if I count by my inbox.> Here''s my system''s configuration: > > eth0: SDSL at 216.15.108.186 (dnai-net) > eth1: ADSL at 67.114.175.138 (sbc-net) > eth2: DHCP network at 192.168.1.1 . > lo: 127.0.0.1 > > There is a firewall box acting as the DHCP server and router for the > systems on the DHCP net. The Linux system doesn''t run DHCP or > masquerading.So what IP address do packets have that come from the firewall box?> If a connection is accepted on the IP address of one of these interfaces, > I''d like its packets to go out on the same interface. If there is a failure > on the path to eth0, connections to eth1 should continue to work, and vice > versa. But right now, eth0 has the default route and just about every packet > goes out of eth0 regardless of what interface they come in. So, I tried this: > > ip rule add from 216.15.108.186 table dnai-net > ip rule add from 67.114.175.138 table sbc-net > ip route add default via 216.15.108.186 dev eth0 table dnai-net > ip route add default via 67.114.175.138 dev eth1 table sbc-net > > This doesn''t work. I get a destination unreachable if I try to follow either > route. So, what am I doing wrong?You have no default route left for packets which came in over the firewall box, which probably has a 192.168.1.x address? I think that is the number one thing going wrong. Now, as to what you want, you will need a way to separate connections to go out over either eth0 or eth1 in a persistent way. This mostly means sending out half your internal IP range over one interface and the other half over the other. This in turn implies SNAT or masquerading because you need to change the source address - packets with the address of eth0 won''t travel the net well over eth1. Regards, bert hubert -- http://www.PowerDNS.com Versatile DNS Software & Services http://www.tk the dot in .tk http://lartc.org Linux Advanced Routing & Traffic Control HOWTO
Bert Hubert:> So what IP address do packets have that come from the firewall box?Oops, I didn''t give enough information. The Linux box is not a router - it''s a multi-homed server. The DHCP net has one of those retail router boxes that does DHCP, NAT, and is a gateway to one of the DSL networks. If I try to follow a policy route to either of the DSL interfaces from the Linux system itself, not _through_ it as a router. ip rule add from 216.15.108.186 table dnai-net ip rule add from 67.114.175.138 table sbc-net ip route add default via 216.15.108.186 dev eth0 table dnai-net ip route add default via 67.114.175.138 dev eth1 table sbc-net ping -I 67.114.175.138 www.gnu.org I get an IMAP "unreachable" message back from the interface. What I want is to be able to bind to the IP address of either of the DSL interfaces, and have the packets routed to that interface reliably. If there''s a network partition effecting one DSL carrier and not the other, it should still be possible to reach the system and the return packets should be on the same network as the incoming ones. Load-balancing is secondary, but I have some ideas there. For example, why not have squid alternate addresses so that receive data is interleaved across the two DSL lines? Wouldn''t that balance better than simply routing half of the internet through each interface? Thanks Bruce
On Sat, Mar 02, 2002 at 09:36:59PM -0800, Bruce Perens wrote:> Bert Hubert: > > So what IP address do packets have that come from the firewall box? > > Oops, I didn''t give enough information. The Linux box is not a router - > it''s a multi-homed server. The DHCP net has one of those retail router > boxes that does DHCP, NAT, and is a gateway to one of the DSL networks.Ah ok, so we have the following: eth0 SDSL --- | Linux -------- [router appliance] ---- ADSL -internet | eth1 --- eth2 DHCP> If I try to follow a policy route to either of the DSL interfaces from the > Linux system itself, not _through_ it as a router. > > ip rule add from 216.15.108.186 table dnai-net > ip rule add from 67.114.175.138 table sbc-net > ip route add default via 216.15.108.186 dev eth0 table dnai-net > ip route add default via 67.114.175.138 dev eth1 table sbc-net > > ping -I 67.114.175.138 www.gnu.org > > I get an IMAP "unreachable" message back from the interface.I still lack information - what are the IP addresses of the Linux machine? Right now you are telling the kernel to route packets FROM 216.15.108.186 TO 216.15.108.186, which isn''t happening :-)> What I want is to be able to bind to the IP address of either of the DSL > interfaces, and have the packets routed to that interface reliably. > If there''s a network partition effecting one DSL carrier and not the other, > it should still be possible to reach the system and the return packets should > be on the same network as the incoming ones.You should make sure the policy routes are made from the IP address of eth0 and eth1, and go to the addresses of the routers there. In that case, ping -I should work. Right now you aren''t routing anything to anything, leading to an ICMP unreachable.> Load-balancing is secondary, but I have some ideas there. For example, why not > have squid alternate addresses so that receive data is interleaved across the > two DSL lines? Wouldn''t that balance better than simply routing half of > the internet through each interface?It isn''t persistent. Many internet sites rely on the fact that users keep coming from the same IP address during their session. Regards, bert -- http://www.PowerDNS.com Versatile DNS Software & Services http://www.tk the dot in .tk http://lartc.org Linux Advanced Routing & Traffic Control HOWTO
On Sun, Mar 03, 2002 at 12:16:54PM +0100, bert hubert wrote:> Right now you are telling the kernel to route packets FROM 216.15.108.186 TO > 216.15.108.186, which isn''t happening :-)I should have known better than _that_. OK, it works now. Thanks very much.>> why not have squid alternate addresses so that receive data is interleaved >> across the two DSL lines? Wouldn''t that balance better than simply routing >> half of the internet through each interface?> It isn''t persistent. Many internet sites rely on the fact that users keep > coming from the same IP address during their session.Good point. I don''t know how, or if, proxies like squid use persistent connections. I suspect that some load-balancing schemes already break them. apt-get (Debian''s package downloader) already knows how to parallelize hosts. It would be nice to parallelize the interfaces at the same time. FYI, this is a picture of my net. 768K SDSL ------------------------------------ eth0 1.5M..384K/128K ADSL ------------------------- eth1 ^ | (10MBPS) | Retail Router Box Linux Box Serves local DHCP hosts with DHCP, NAT, "firewall", gateway. ^ | | Local DHCP net ------------------------------- eth2 All networks are 100/10 MBit full-duplex switches. Both DSL modems and the uplink port of the retail router run 10 MBit half-duplex, everything else runs 100 MBit full-duplex. Not much should be going through the retail router box, as the Linux box serves the local net with squid, IMAP, SMTP, and telnet. I could dispense with the router box and route with Linux if I wanted to. Attached is what I ended up with in /etc/network/interfaces: Thanks Bruce auto eth0 iface eth0 inet static address 216.15.108.186 network 216.15.108.184 netmask 255.255.255.248 broadcast 216.15.108.255 # gateway 216.15.108.185 up ip rule add from 216.15.108.186 table dnai-net up ip route add default via 216.15.108.185 dev eth0 table dnai-net up ip route add default via 216.15.108.185 dev eth0 metric 1 table main down ip rule del from 216.15.108.186 table dnai-net down ip route del default via 216.15.108.185 dev eth0 table dnai-net down ip route del default via 216.15.108.185 dev eth0 metric 1 table main auto eth1 iface eth1 inet static address 67.114.175.138 network 67.114.175.137 netmask 255.255.255.248 broadcast 67.114.175.143 up ip rule add from 67.114.175.138 table sbc-net up ip route add default via 67.114.175.137 dev eth1 table sbc-net up ip route add default via 67.114.175.137 dev eth1 metric 2 table main down ip rule del from 67.114.175.138 table sbc-net down ip route del default via 67.114.175.137 dev eth1 table sbc-net down ip route del default via 67.114.175.137 dev eth1 metric 2 table main auto eth2 iface eth2 inet static address 192.168.1.1 network 192.168.1.0 netmask 255.255.255.0 broadcast 192.168.1.255