LARTC - Sep 2005 - iproute2/nano-howto: dual external routing, a "virtual subnet"

A site I administer has dual T1''s. We have this set up according to 
NANO-HOWTO[1] using Julian''s routes patch on a late model 2.4 kernel. 
All is well, has been working well for a long time.

Recently one of our T1 providers went belly up and we got something 
better. Now we have a /28 where previously we only used one IP per 
interface.

Present interfaces:
eth0 - T1#1, single IP, x.x.x.52/24 (gw x.x.x.1)
eth1 (Gigabit) - internal / trusted LAN, 192.168.27.1
eth2 - isolated/restricted access LAN (including wireless) 192.168.37.1
hdlc0 - T1#2, y.y.y.54/30 (gw y.y.y.53)
[    new netblock, y.y.z.144/28, is routed to y.y.y.54   ]
tunX - numerous OpenVPN point-to-point tunnels, 192.168.4.x
pppX - pptpd/pppd for external MS Windows users (due to be replaced
       by an openvpn server), 192.168.26.x
eth1 and hdlc0 are set up as dual gateways, equal weight, following 
nano-howto to the letter (because I didn''t know any better.)

The plan is to use the new /28 for a mix of virtual user-mode Linux 
installs and openvpn endpoints to offsite peers. What we now have as 
192.168.4.x will be converted to y.y.z.144/28. It will be similar to a 
DMZ in some aspects.

I''m not sure how to do that without breaking the routing we have now. 
First thing I''m trying is to bind the /28 to dummy0.

# ip ad ls
145: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:04:75:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet x.x.x.52/24 brd x.x.x.255 scope global eth0
146: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:02:b3:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet 192.168.27.1/24 brd 192.168.27.255 scope global eth1
147: hdlc0: <POINTOPOINT,NOARP,UP> mtu 1500 qdisc pfifo_fast qlen 100
    link/hdlc
    inet y.y.y.54 peer y.y.y.53/32 scope global hdlc0
148: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:04:5a:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet 192.168.37.1/24 brd 192.168.37.255 scope global eth2
843: dummy0: <BROADCAST,NOARP,UP> mtu 1500 qdisc noqueue
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
    inet y.y.z.145/28 scope global dummy0
844: tun0: <POINTOPOINT,MULTICAST,NOARP,UP> mtu 1400 qdisc pfifo_fast 
qlen 10
    link/ppp
    inet 192.168.6.46 peer y.y.z.146/32 scope global tun0
[superfluous interfaces omitted]

# ip ru ls
0:      from all lookup local
50:     from all lookup main
201:    from x.x.x.0/24 lookup 201
202:    from y.y.y.54 lookup 202
222:    from all lookup 222
32766:  from all lookup main
32767:  from all lookup default

# ip ro ls
y.y.y.53 dev hdlc0  proto kernel  scope link  src y.y.y.54
y.y.z.146 dev tun0  proto kernel  scope link  src 192.168.6.46
y.y.z.144/28 dev dummy0  proto kernel  scope link  src y.y.z.145
192.168.27.0/24 dev eth1  proto kernel  scope link  src 192.168.27.1
192.168.37.0/24 dev eth2  proto kernel  scope link  src 192.168.37.1
x.x.x.0/24 dev eth0  proto kernel  scope link  src x.x.x.52
[superfluous routes omitted]

Success!! I played around with this off and on all day, and now it 
works. I turned on /proc/sys/net/ipv4/conf/dummy0/proxy_arp (how can 
the interface be flagged NOARP and yet do proxy ARP?) on that end and 
set up a route table on the other end:
    echo "32 vpn" >> /etc/iproute2/rt_tables
    ip ru add add from y.y.z.146 table vpn
    ip ro add default via 192.168.6.46 table vpn
The services which are exposed in the firewall are accessible through 
y.y.z.146. Very nice!

But I''m still not sure about some things. I want to understand better. 
I''m also not sure it will work right. A lot of this is new to me, so 
please bear with me.

Is the use of dummy0 needed? ISTM it was just a simple way to get the 
routing tables set up. And it''s probably wasting 3 IP''s: .144 
network, .145 dummy0, and .159 broadcast. With these IP''s all coming 
through y.y.y.54 as they are, is there any reason why I can''t use the 
network and broadcast addresses for (virtual) hosts? With proper routes 
and an "ip neigh add ..." command I shouldn''t need dummy0
here, right?
But I don''t know what to do.

Do I need a similar route table on the dual-T1 end? Do I need to do 
something to force y.y.z.144/28 out hdlc0? I guess it might not matter 
depending on the eth0 ISP''s upstream router x.x.x.1; if it lets the 
packets from the hdlc0 ISP''s netblocks pass, no one downstream is 
likely to know or care, right?

Numerically, where should such a table go?

And now, here''s the biggie ... if we got a netblock from the other ISP,
could we set up these virtual hosts to bind to one IP from each ISP, 
and thus gain the redundancy?

Having lots of fun; I hope you are too. :)



[1] http://www.ssi.bg/~ja/nano.txt
-- 
    mail to this address is discarded unless "/dev/rob0"
    or "not-spam" is in Subject: header