Hi LARTC,
I have two networks that I manage, A and B. They both have their own
primary gateways for hosts on the network, let''s call them AR and BR.
There is another box, A1, which lives on network A but also has a wire
connecting it to a switch on network B. My goal is to let hosts on B
access three particular hosts on A''s subnet (192.168.4.0/24) and let
hosts on A access one particular host on B''s subnet (10.3.0.0/24).
Suppose the IP addresses of the machines I wish to address on either
network are A2, A3, A4 and B1.
A1 has IP 10.3.0.13 on network B and 192.168.4.10 on network A. Is it
possible to just tell the gateways for each network:
on BR: ip route add {A2, A3, A4} via 10.3.0.13
on AR: ip route add B1 via 192.168.4.10
But when I do that, ping does something like:
PING A2 56(84) bytes of data.
From BR: Redirect Host (new nexthop: 10.3.0.13)
and doesn''t actually report success on the ping.
What am I doing wrong? Is this a job for iptables? I feel like I can do
this with just routing table magic.
Thanks,
--
Joshua Kwan
Joshua Kwan wrote:> Hi LARTC, > > I have two networks that I manage, A and B. They both have their own > primary gateways for hosts on the network, let''s call them AR and BR. > There is another box, A1, which lives on network A but also has a wire > connecting it to a switch on network B. My goal is to let hosts on B > access three particular hosts on A''s subnet (192.168.4.0/24) and let > hosts on A access one particular host on B''s subnet (10.3.0.0/24). > > Suppose the IP addresses of the machines I wish to address on either > network are A2, A3, A4 and B1. > > A1 has IP 10.3.0.13 on network B and 192.168.4.10 on network A. Is it > possible to just tell the gateways for each network: > > on BR: ip route add {A2, A3, A4} via 10.3.0.13 > on AR: ip route add B1 via 192.168.4.10 > > But when I do that, ping does something like: > > PING A2 56(84) bytes of data. > From BR: Redirect Host (new nexthop: 10.3.0.13)That''s to be expected, as far as I understand.> and doesn''t actually report success on the ping.> What am I doing wrong? Is this a job for iptables? I feel like I can do > this with just routing table magic.Well, nobody has given you a better answer yet, so I''ll take a shot. 1. Check /proc/sys/net/ipv4/ip_forward on box A1 to see if it''s configured to route at all. 2. The hosts on network A will receive packets from miscellaneous IP addresses in B''s subnet, and not have any idea what to do with them. You''ll need to either configure routes on the hosts in network A... # ip route add 10.3.0.0/16 via 192.168.4.10 dev eth0 ...or set up box A1 to SNAT packets # iptables -t nat -A POSTROUTING -s 10.3.0.0/16 -d 192.168.4.0/24 \ -j SNAT --to-source 192.168.4.10 You''ll have to do something similar for the reverse direction, too. I haven''t tested what I wrote above; if it doesn''t work, run tcpdump in various places to see what icmp packets are going where. Honestly, though, I think you''re setting yourself up for a decent headache trying to do it this way. In my opinion, the easy solution is in the first word of your subject line: just make host A1 a bridge. That''s what I did when I originally set up that box, and it worked fine the whole time I managed it (actually, I didn''t manage it at all, since the bridging never had any problems). Put both physical networks on the same IP subnet and enforce any desired firewalling/segmentation with a few simple iptables rules running on the bridge. If you''re still using the same machine that used to do that job, you might be able to find the relevant configuration files lying around. If not, I''m sure I can recall at least most of the details. Either way, if you need assistance, let me know and I''ll be happy to stop by sometime and lend a hand. -Corey
On 10/31/07 19:50, Joshua Kwan wrote:> What am I doing wrong? Is this a job for iptables? I feel like I can > do this with just routing table magic.As Corey said, make sure that IP Forwarding is enabled on A1. You are setting your self up for a very weird scenario. Probably what would be the easiest for long term would be to put a third network between your two routers, AR and BR. This way you end up with the following type of scenario: +----+ +----+ | AI | | BI | +-+--+ +--+-+ | | +-+--+ +--+-+ ---( Net A )---+ AR +---( Net C )---+ BR +---( Net B )--- | +----+ +----+ | | | | +----+ | +-------+ A1 +----------------------------+ +----+ With this type of set up, you can configure both AR and BR that each can reach the others network via the other router on network C. I.e. AR can reach Net B via BR''s IP on Net C. Likewise BR can reach Net A via AR''s IP on Net C. This way, each and every computer on both networks A and B can reach any host they want to via their default gateway AR or BR respectively. This may be a little more work to set up initially, but it is very simple routing that will be very easy to maintain for a long time. In this scenario Net C can very simply be a cross over cable between routers AR and BR. Another option is to enable routing on A1 and tell A2, A3, and A4 that they can reach B1 via A1 and likewise tell B1 that it can reach a2, A3, and A4 via A1. Thus you have a bi-directional routing set up between just the four additional hosts. The main disadvantage of this set up is that each and every host that wants to communicate across networks will have to have its own routing entries set up to pick the correct direction to route traffic to. So if you only have two or three hosts on each network and you can not do the additional network (C) between the two routers, this will work. There are some other things that you can do to allow the hosts to communicate across the networks like bridging and / or proxy ARP. However seeing as how you have different subnets on Net A and Net B, you will very likely be doing routing or multi-homing on all hosts to support bridging and / or proxy ARP. I think your best bet is to do the additional network (C) between the two routers AR and BR. Grant. . . .
On 11/02/07 15:24, Corey Hickey wrote:> 1. Check /proc/sys/net/ipv4/ip_forward on box A1 to see if it''s > configured to route at all. > > 2. The hosts on network A will receive packets from miscellaneous IP > addresses in B''s subnet, and not have any idea what to do with them. > You''ll need to either configure routes on the hosts in network A... > > # ip route add 10.3.0.0/16 via 192.168.4.10 dev eth0 > > ...or set up box A1 to SNAT packets > > # iptables -t nat -A POSTROUTING -s 10.3.0.0/16 -d 192.168.4.0/24 -j > SNAT --to-source 192.168.4.10 > > You''ll have to do something similar for the reverse direction, too.You could do either of the above. However there should be a route from any box on either network to specific boxes on the other network. Remember that the OP configured routes on AR and BR to reach host(s) on Net B and Net A (respectively) via A1. Thus when a host on Net B receives a packet from a host on Net A, it will reply via its default gateway, BR. BR will then redirect or route the packet back to A1 which will then send the packet directly on to the original host on Net A. This is all of course presuming that A1 is forwarding packets like it needs to be and that there are no firewalls in place blocking any thing, especially reverse path filters.> I haven''t tested what I wrote above; if it doesn''t work, run tcpdump > in various places to see what icmp packets are going where.TCPDump / Etherial is your *FRIEND*!!!> Honestly, though, I think you''re setting yourself up for a decent > headache trying to do it this way. In my opinion, the easy solution > is in the first word of your subject line: just make host A1 a > bridge. That''s what I did when I originally set up that box, and it > worked fine the whole time I managed it (actually, I didn''t manage it > at all, since the bridging never had any problems).Bridging and / or proxy ARP by them selves will not solve this problem. Net A and Net B have different IP address ranges, thus even with a bridge in place the hosts will never communicate. In fact a bridge with out any thing else will just worsen the situation by passing broadcasts where they are not needed.> Put both physical networks on the same IP subnet and enforce any > desired firewalling/segmentation with a few simple iptables rules > running on the bridge. If you''re still using the same machine that > used to do that job, you might be able to find the relevant > configuration files lying around. If not, I''m sure I can recall at > least most of the details.Ugh. I''m going to presume that the networks have different IP address ranges and are not presently joined for a reason. Thus combining the IP address ranges and bridging them together will actually be a regression in network config / technology / evolution. Now if you want to bridge the networks together wile still using the separate IP address ranges AND multi-home select boxen so that they are on both Net A and Net B then bridging and / or proxy ARP will work just fine.> Either way, if you need assistance, let me know and I''ll be happy to > stop by sometime and lend a hand.You can drop a line in here too. Grant. . . .
Grant Taylor wrote:> On 11/02/07 15:24, Corey Hickey wrote: >> 1. Check /proc/sys/net/ipv4/ip_forward on box A1 to see if it''s >> configured to route at all. >> >> 2. The hosts on network A will receive packets from miscellaneous IP >> addresses in B''s subnet, and not have any idea what to do with them. >> You''ll need to either configure routes on the hosts in network A... >> >> # ip route add 10.3.0.0/16 via 192.168.4.10 dev eth0 >> >> ...or set up box A1 to SNAT packets >> >> # iptables -t nat -A POSTROUTING -s 10.3.0.0/16 -d 192.168.4.0/24 -j >> SNAT --to-source 192.168.4.10 >> >> You''ll have to do something similar for the reverse direction, too. > > You could do either of the above.I meant to do both, which I think is necessary in order to make the OPs proposed scheme work without modification. I''ll defer if I''m wrong, though--I haven''t tested it, and, as you said in your other email, it''s "a very weird scenario."> However there should be a route from any box on either network to > specific boxes on the other network. Remember that the OP configured > routes on AR and BR to reach host(s) on Net B and Net A (respectively) > via A1. Thus when a host on Net B receives a packet from a host on Net > A, it will reply via its default gateway, BR. BR will then redirect or > route the packet back to A1 which will then send the packet directly on > to the original host on Net A. This is all of course presuming that A1 > is forwarding packets like it needs to be and that there are no > firewalls in place blocking any thing, especially reverse path filters.I don''t think this will work unless BR has a route like: # ip route add 192.168.4.0/24 via 10.3.0.13 dev eth0 ...whereas the OP only specified wanting routes to a few specific machines rather than the whole networks. In any case, debating that is probably academic, since I agree with you in principle. It should be cleaner to set up routes for the whole networks and use iptables rules on A1 to only allow traffic to/from specified hosts. There are certainly different ways to do it, and I furthermore agree with you that using a separate link between AR and BR (as you suggested in your earlier message) is cleaner still. I prefer bridging in this situation mostly because it distributes traffic and reduces the load on the routers.>> I haven''t tested what I wrote above; if it doesn''t work, run tcpdump >> in various places to see what icmp packets are going where. > > TCPDump / Etherial is your *FRIEND*!!!:)>> Honestly, though, I think you''re setting yourself up for a decent >> headache trying to do it this way. In my opinion, the easy solution >> is in the first word of your subject line: just make host A1 a >> bridge. That''s what I did when I originally set up that box, and it >> worked fine the whole time I managed it (actually, I didn''t manage it >> at all, since the bridging never had any problems). > > Bridging and / or proxy ARP by them selves will not solve this problem. > Net A and Net B have different IP address ranges, thus even with a > bridge in place the hosts will never communicate. In fact a bridge with > out any thing else will just worsen the situation by passing broadcasts > where they are not needed.Absolutely.>> Put both physical networks on the same IP subnet and enforce any >> desired firewalling/segmentation with a few simple iptables rules >> running on the bridge. If you''re still using the same machine that >> used to do that job, you might be able to find the relevant >> configuration files lying around. If not, I''m sure I can recall at >> least most of the details. > > Ugh. I''m going to presume that the networks have different IP address > ranges and are not presently joined for a reason. Thus combining the IP > address ranges and bridging them together will actually be a regression > in network config / technology / evolution.The two networks in question are rather small and occupy adjoining buildings. Network A had to be rebuilt after getting torn out while the corresponding building underwent a very intrusive retrofit and remodeling. Prior to that, the two networks were bridged and shared the same subnet. I don''t know if the OP has a reason to isolate them from each other now. I guess I''ll go ahead and describe the former setup in a little detail. Every host in the entire bridged network was given an IP address within the subnet 10.0.0.0/8. The bridge was configured to drop all DHCP packets, so there was a DHCP server on network A and another on network B. Hosts on network A were given addresses in the following ranges[1]: 10.0.0.0/16 10.1.0.0/16 10.2.0.0/16 Hosts on network B were given addresses in the following range: 10.3.0.0/16 ...but, regardless of which network a host was on, it still was given the /8 subnet, so hosts could communicate over the bridge without any further configuration. Since each network had its own router to the Internet, the DHCP servers also specified separate gateways. The bridge was configured to drop packets with sources or destinations that didn''t match the IP ranges corresponding to the source/destination networks[2]. That''s all. My philosophy was to allow unrestricted communication over the bridge and gently LART users that caused trouble (always inadvertently; Windows worms and such). If the OP wants to allow communication only to a few hosts, that''s no more difficult--just write a few rules to accept desired traffic and then drop/reject the rest. [1] Given the chance to do it over, I would have allocated addresses to network A from 10.0.0.0/18 and network B from 10.4.0.0/18 in order to simplify a little bit. Also, I should mention that the use of several /16 ranges doesn''t mean we had anywhere near that many hosts; the separation was just for management. [2] Just in case some users on network B tried to manually set their IP address and gateway in order to use the better Internet access of network A. Of course, they could still have tunneled through the bridge to an accomplice on network A, but they could have also used an accomplice''s wireless router, or CAT-5 strung between rooftops, or RFC 1149, etc. I dealt with such things on a case-by-case basis. :) -Corey
On 11/2/2007 11:35 PM, Corey Hickey wrote:> I meant to do both, which I think is necessary in order to make the > OPs proposed scheme work without modification. I''ll defer if I''m > wrong, though--I haven''t tested it, and, as you said in your other > email, it''s "a very weird scenario."As long as there are routes in both directions there should be no need for SNATing.> I don''t think this will work unless BR has a route like: > > # ip route add 192.168.4.0/24 via 10.3.0.13 dev eth0 > > ...whereas the OP only specified wanting routes to a few specific > machines rather than the whole networks. > > In any case, debating that is probably academic, since I agree with > you in principle. It should be cleaner to set up routes for the whole > networks and use iptables rules on A1 to only allow traffic to/from > specified hosts.Agreed. I mis read the routes on the two routers AR and BR to be for the entire networks. Though again presuming there are routes, things should work. This is more just a semantical mis-interpretation on the scope of what the routes are for.> There are certainly different ways to do it, and I furthermore agree > with you that using a separate link between AR and BR (as you > suggested in your earlier message) is cleaner still. > > I prefer bridging in this situation mostly because it distributes > traffic and reduces the load on the routers.I can see how this would reduce load on the routes, but I don''t believe that load on routers will be much of a concern. (At least the routers that I use (pick any box (less than 10 years old) and install Linux) would do just fine. However I would be concerned about broadcast storms being propagated across the bridge unnecessarily. But if steps are taken to mitigate that then it is probably not that big of an issue.> The two networks in question are rather small and occupy adjoining > buildings. Network A had to be rebuilt after getting torn out while > the corresponding building underwent a very intrusive retrofit and > remodeling. Prior to that, the two networks were bridged and shared > the same subnet. I don''t know if the OP has a reason to isolate them > from each other now.Ok... Obviously you are probably in a very unique position knowing the history of the network.> I guess I''ll go ahead and describe the former setup in a little > detail. > > Every host in the entire bridged network was given an IP address > within the subnet 10.0.0.0/8. The bridge was configured to drop all > DHCP packets, so there was a DHCP server on network A and another on > network B.Ok...> Hosts on network A were given addresses in the following ranges[1]: > > 10.0.0.0/16 > 10.1.0.0/16 > 10.2.0.0/16 > > Hosts on network B were given addresses in the following range: > > 10.3.0.0/16 > > ...but, regardless of which network a host was on, it still was given > the /8 subnet, so hosts could communicate over the bridge without any > further configuration.Ok, you chose to do in bridging what most people do in routing. Seeing as how things were bridged you had to put things in place to stop things that would naturally leave the subnet. Your preference to have and work with.> Since each network had its own router to the Internet, the DHCP > servers also specified separate gateways. The bridge was configured > to drop packets with sources or destinations that didn''t match the IP > ranges corresponding to the source/destination networks[2].Ok...> That''s all.So let me get this right, you did bridging rather than routing to avoid load on the router(s)? Yet you had to put more load on the bridging host to segregate the networks like they would be if they were routed while still allowing host to host communications between the two buildings? <personal opinion> Strange </personal opinion>> My philosophy was to allow unrestricted communication over the bridge > and gently LART users that caused trouble (always inadvertently; > Windows worms and such). If the OP wants to allow communication only > to a few hosts, that''s no more difficult--just write a few rules to > accept desired traffic and then drop/reject the rest.Ok.> [1] Given the chance to do it over, I would have allocated addresses > to network A from 10.0.0.0/18 and network B from 10.4.0.0/18 in order > to simplify a little bit. Also, I should mention that the use of > several /16 ranges doesn''t mean we had anywhere near that many hosts; > the separation was just for management.*nod*> [2] Just in case some users on network B tried to manually set their > IP address and gateway in order to use the better Internet access of > network A. Of course, they could still have tunneled through the > bridge to an accomplice on network A, but they could have also used > an accomplice''s wireless router, or CAT-5 strung between rooftops, or > RFC 1149, etc. I dealt with such things on a case-by-case basis. :)That''s what a "Clue-by-4" is used for. ;) All in all you chose to implement a solution in one way that very like did exactly what you needed even if it was a bit different than what the industry norm would have been. Either way, bridging or routing, they both would have / do / will work. Grant. . . .