I''ve been looking for about a year to find a way to use more than one independent link to the Internet from a single computer. But I only found many others who tried to do the same, without success. Then I found out about Julian Anastasov''s patches, and well, now it''s working. Unfortunately, this is not a five minutes solution, nor will it help in all kinds of situations. So I decided to write a document which tells step by step how this can be done and how it works, for which I received lots of support from Julian. Thanks. I''ve Attached the text to this message. I hope it is useful to others. Here it is already working. -- Christoph Simon ciccio@kiosknet.com.br --- ^X^C q quit :q ^C end x exit ZZ ^D ? help .
I''ve been looking for about a year to find a way to use more than one independent link to the Internet from a single computer. But I only found many others who tried to do the same, without success. Then I found out about Julian Anastasov''s patches, and well, now it''s working. Unfortunately, this is not a five minutes solution, nor will it help in all kinds of situations. So I decided to write a document which tells step by step how this can be done and how it works, for which I received lots of support from Julian. Thanks. The text can be found at: http://www.linuxvirtualserver.org/~julian/nano.txt -- Christoph Simon ciccio@kiosknet.com.br --- ^X^C q quit :q ^C end x exit ZZ ^D ? help .
Thanks Christoph (and Julian!), by happy coincidence this is exactly what I''m looking for today. In nano.txt you say the firewall, for iptables, must be stateful. Of course, ipchains doesn''t do stateful. I''m looking at using Julian''s patches with a 2.2.20 kernel and ipchains and masquerading. Does anyone know offhand whether I should: 1. Expect this to work? 2. Expect this to get weird? If 2: - What weirdness should I look out for? - What, in theory, is the statefulness accomplishing in this context? Whit On Sun, Dec 02, 2001 at 05:41:54PM -0200, Christoph Simon wrote:> I''ve been looking for about a year to find a way to use more than one > independent link to the Internet from a single computer. But I only > found many others who tried to do the same, without success. > > Then I found out about Julian Anastasov''s patches, and well, now it''s > working. Unfortunately, this is not a five minutes solution, nor will > it help in all kinds of situations. So I decided to write a document > which tells step by step how this can be done and how it works, for > which I received lots of support from Julian. Thanks. > > The text can be found at: > > http://www.linuxvirtualserver.org/~julian/nano.txt
On Mon, 3 Dec 2001 15:22:25 -0500 Whit Blauvelt <whit@transpect.com> wrote:> Thanks Christoph (and Julian!), by happy coincidence this is exactly what > I''m looking for today. > > In nano.txt you say the firewall, for iptables, must be stateful. Of course, > ipchains doesn''t do stateful. I''m looking at using Julian''s patches with a > 2.2.20 kernel and ipchains and masquerading. Does anyone know offhand > whether I should: > > 1. Expect this to work? > > 2. Expect this to get weird? > > If 2: > > - What weirdness should I look out for? > > - What, in theory, is the statefulness accomplishing in this context?Honestly, I have no idea. Maybe Julian can help you out. I''ve tested this only on 2.4 with iptables. The setup isn''t a three mouseclick action, but it''s also not that bad, that you couldn''t just try. -- Christoph Simon ciccio@kiosknet.com.br --- ^X^C q quit :q ^C end x exit ZZ ^D ? help .
On Mon, 3 Dec 2001, Whit Blauvelt wrote:> Thanks Christoph (and Julian!), by happy coincidence this is exactly what > I''m looking for today.> In nano.txt you say the firewall, for iptables, must be stateful. Of course, > ipchains doesn''t do stateful. I''m looking at using Julian''s patches with a > 2.2.20 kernel and ipchains and masquerading. Does anyone know offhand > whether I should:> 1. Expect this to work?Yes.> 2. Expect this to get weird?Maybe. But that''s not related to 2.2, it can happen with 2.4 as well.> If 2: > > - What weirdness should I look out for?OpenSSH sets up the TOS fields *after* authenticating. This breaks the entries in the route-cache, as they are keyed on source, destination and TOS field.> - What, in theory, is the statefulness accomplishing in this context?Don''t really know, as I haven''t needed it. (I''ve set up a similar system with only 2.2, never even so much as thinking about 2.4). Doei, Arthur. (Oh... in my opinion the firewalling is an optional extra.) -- /\ / | arthurvl@sci.kun.nl | Work like you don''t need the money /__\ / | A friend is someone with whom | Love like you have never been hurt / \/__ | you can dare to be yourself | Dance like there''s nobody watching
On Mon, 3 Dec 2001 23:45:49 +0000 (GMT) Julian Anastasov <ja@ssi.bg> wrote:> On Mon, 3 Dec 2001, Whit Blauvelt wrote: > > > Thanks Christoph (and Julian!), by happy coincidence this is exactly what > > I''m looking for today. > > > > In nano.txt you say the firewall, for iptables, must be stateful. Of course, > > ipchains doesn''t do stateful. I''m looking at using Julian''s patches with a > > Assume that this is recommendation (should).I was reviewing my notes and now I believe, that statefulness should not strictly be required, though at least for netfilter, I think it is very recommendable. What is required is connection tracking, as Julian''s patches use their information for routing decisions. If you use the netfilter snat target, actually it shouldn''t make a difference, as far as I can see. Maybe there is one, when using the masquerade target, because there might be a second call to the routing system, but I tend to believe that not. It''s possible that I''m completely wrong, but I think it''s definitively worth a try. -- Christoph Simon ciccio@kiosknet.com.br --- ^X^C q quit :q ^C end x exit ZZ ^D ? help .
On Mon, Dec 03, 2001 at 11:04:20PM +0100, Arthur van Leeuwen wrote:> > - What weirdness should I look out for? > > OpenSSH sets up the TOS fields *after* authenticating. This breaks the > entries in the route-cache, as they are keyed on source, destination and TOS > field.Hmm, I use OpenSSH all the time. Does this fully break it, or just add some flakiness? Thanks again, Whit
Hi Julian, It looks like your patch is not compatible with FreeS/WAN''s (1.92) patches. Or at least in trying to compile a 2.2.20 kernel, it works fine with yours, but then adding FreeS/WAN leads to errors. Word has it that FreeS/WAN 1.92 normally works with 2.2.20, but haven''t verified that by experiment. Guess it makes sense that you''d both be getting into some of the same sections of kernel code. I''m not saying this is something you should fix - I''m not all that impressed with FreeS/WAN as an implementation (actually the commercial IPSec implementations I''ve tested are also pretty flakey - hope that protocol grows up some day). Just wanted to note to folks that there seems to be an incompatibility here. In the long run, encrypted communications and good redundant routing are both vital services. Whit @transpect.com
Hello, On Mon, 3 Dec 2001, Whit Blauvelt wrote:> Thanks Christoph (and Julian!), by happy coincidence this is exactly what > I''m looking for today. > > In nano.txt you say the firewall, for iptables, must be stateful. Of course, > ipchains doesn''t do stateful. I''m looking at using Julian''s patches with aAssume that this is recommendation (should).> 2.2.20 kernel and ipchains and masquerading. Does anyone know offhand > whether I should: > > 1. Expect this to work?If the settings are correct and I didn''t broke something when building all pieces together. The end goal of the patches both for 2.2 and 2.4 should be same. The implementations differ, Netfilter is more suitable for such changes while 2.2 has some weirdness supporting these extensions. The same weirdness you can see in the changes for the ipchains compat code in 2.4.> 2. Expect this to get weird? > > If 2: > > - What weirdness should I look out for?Make some tests before going to production :) It needs some understanding. That is why the document Christoph wrote is so useful.> - What, in theory, is the statefulness accomplishing in this context?If I understand your question correctly, this is not a goal. It is a conntracking specific thing which is not touched from these patches. These patches change the routing and the way it is used from NAT after adding or extending some nice features.> WhitRegards -- Julian Anastasov <ja@ssi.bg>
Hello, On Mon, 3 Dec 2001, Whit Blauvelt wrote:> On Mon, Dec 03, 2001 at 11:04:20PM +0100, Arthur van Leeuwen wrote: > > > > - What weirdness should I look out for? > > > > OpenSSH sets up the TOS fields *after* authenticating. This breaks the > > entries in the route-cache, as they are keyed on source, destination and TOS > > field. > > Hmm, I use OpenSSH all the time. Does this fully break it, or just add some > flakiness?It breaks only the plain kernel. The patches use the multipath routes only for the first packet.> Thanks again, > WhitRegards -- Julian Anastasov <ja@ssi.bg>
Hello, On Mon, 3 Dec 2001, Whit Blauvelt wrote:> Hi Julian, > > It looks like your patch is not compatible with FreeS/WAN''s (1.92) patches.It is possible, I didn''t tried.> Or at least in trying to compile a 2.2.20 kernel, it works fine with yours, > but then adding FreeS/WAN leads to errors. Word has it that FreeS/WAN 1.92 > normally works with 2.2.20, but haven''t verified that by experiment. Guess > it makes sense that you''d both be getting into some of the same sections of > kernel code. > > I''m not saying this is something you should fix - I''m not all that impressed > with FreeS/WAN as an implementation (actually the commercial IPSec > implementations I''ve tested are also pretty flakey - hope that protocol > grows up some day). Just wanted to note to folks that there seems to be an > incompatibility here. In the long run, encrypted communications and good > redundant routing are both vital services.OK, I''ll try in the next days to see what is the issue.> Whit > @transpect.comRegards -- Julian Anastasov <ja@ssi.bg>
On Tue, 4 Dec 2001, Julian Anastasov wrote:> > Hello, > > On Mon, 3 Dec 2001, Whit Blauvelt wrote: > > > On Mon, Dec 03, 2001 at 11:04:20PM +0100, Arthur van Leeuwen wrote: > > > > > > - What weirdness should I look out for? > > > > > > OpenSSH sets up the TOS fields *after* authenticating. This breaks the > > > entries in the route-cache, as they are keyed on source, destination and TOS > > > field. > > > > Hmm, I use OpenSSH all the time. Does this fully break it, or just add some > > flakiness? > > It breaks only the plain kernel. The patches use the > multipath routes only for the first packet.Okay, I''ve read both the nanohowto and the docs on Julian''s patches by now. A few things to note: the nanohowto''s information is good even without Julian''s patches, although things will become trickier. One has to do ones own link-probing and rerouting from userland. That is very doable however, provided you have machines somewhere at the ISP''s site that will answer to either pings ore traceroutes or somesuch, as you will need answers. The patches Julian provided fix a bunch of nastiness. For one, dead gateway detection is done on the ARP level in kernelspace. Very neat when you have ARP, thus on ethernet, but not very useful without. Furthermore, they provide true alternative routes, not only multipath default routes. This is once more extremely neat, but not directly necessary for the usual case. Thirdly, Julian''s patches add gateways as a routing key. This will not help pure routing boxes, such as would be standard issue in an office full of Windows toasters, as the gateway will be determined at the routing stage, so it cannot be used as a key. The *main* reason to use Julian''s patches is the masquerading connection rerouting. This will fix the big bugs in your setup by just redirecting a masqueraded connection out to a different interface when the old one is dead. This is *very* cool on UDP, and will make UDP failover to another route fully transparent. However, it will not fix stateful protocols in which the server on the other side keeps state on the IP address it was talking to, such as SSH. It will fix the TOS nastiness OpenSSH brings to the fore, as it will *reroute* after masquerading. Bit of a hack, that. I simply nixed the TOS bits in the firewalling code. :) Summarizing: Yes, you can do equal cost multipath. Yes it is cool. Yes it can be made nicer and friendlier to set up using Julian''s patches. However, it will not be an ideal solution. Things *will* break. Load will just be approximately balanced. Failover is in most cases definitely not transparent to the user: new connections have to be set up. If the links stay up though, equal cost multipath is a *good* thing. Oh, and it does work on >2 uplinks. I''ve set up a system for a client using 1 ISDN line, 2 ADSL links (with the Dutch MXStream cruftiness, but I digress) and 1 cable modem using masquerading only on the last three, using the standard kernel (Julian''s patches didn''t exist a year ago, when I did this). Worked splendidly (and still does, I''m told). Needed some manual supervision though, as link failover and especially failback is *not* trivial. Doei, Arthur. -- /\ / | arthurvl@sci.kun.nl | Work like you don''t need the money /__\ / | A friend is someone with whom | Love like you have never been hurt / \/__ | you can dare to be yourself | Dance like there''s nobody watching
Hello, On Tue, 4 Dec 2001, Arthur van Leeuwen wrote:> Okay, I''ve read both the nanohowto and the docs on Julian''s patches by now. > A few things to note: the nanohowto''s information is good even without > Julian''s patches, although things will become trickier. One has to do ones > own link-probing and rerouting from userland. That is very doable however,what do you mean with "rerouting from userland"?> provided you have machines somewhere at the ISP''s site that will answer to > either pings ore traceroutes or somesuch, as you will need answers.BTW, only ARP answers are needed, they can''t be filtered with an excuse of ISP''s policy. The ICMP replies are used from the userland only (for monitoring). Of course, this is one solution, as you said, one can do the same from user space by changing the routes after a failover. Even from user space it can be faster in some cases as the kernel do passive dead gateway (neighbour) detection, not active.> > The patches Julian provided fix a bunch of nastiness. For one, dead gateway > detection is done on the ARP level in kernelspace. Very neat when you have > ARP, thus on ethernet, but not very useful without. Furthermore, theyRight. But there are non-ARP device drivers that can alter the device link state. For others, it is in the hands of a user program. At least, the patches handle situations with such pointtopoint devices.> provide true alternative routes, not only multipath default routes. This is > once more extremely neat, but not directly necessary for the usual case. > > Thirdly, Julian''s patches add gateways as a routing key. This will not help > pure routing boxes, such as would be standard issue in an office full of > Windows toasters, as the gateway will be determined at the routing stage, so > it cannot be used as a key.The gateway as key has the purpose only for the NAT to select the best source address according to data provided at routing time about the path from the multipath route that is selected for this packet. No other hosts depend on this. It is a way to select a source IP for the link already selected at routing time when you are using paths with same output device in the multipath route. We can distinguish them only by outdev and gateway. The plain kernel matches them only by device with the assumptions that the other variant is insecure.> > The *main* reason to use Julian''s patches is the masquerading connection > rerouting. This will fix the big bugs in your setup by just redirecting a > masqueraded connection out to a different interface when the old one is > dead. This is *very* cool on UDP, and will make UDP failover to another > route fully transparent. However, it will not fix stateful protocols inIt is no protocol specific. As for the established connections, they are already bound to specific source and the number of variants are equal to the number of links that serve this public IP (1 in most of the setups), so, there is no place for failover for the masqueraded connections in the usual setups. As for the plain usage of alternative routes even TCP can start to use them except when the socket is bound to device. In all other cases you can setup subnet over 2 devices and when one fails the TCP connections (with short gc_interval) will switch to the other line (slooowly sometimes).> which the server on the other side keeps state on the IP address it was > talking to, such as SSH. It will fix the TOS nastiness OpenSSH brings to theNobody changes IP addresses, once the connection is created its traffic can go only through the allowed devices for the addresses. The multipath route is used only from the new connections to select different line for them. So, the remote end will not detect packets with changed source. We simply can''t send them through the wrong device. The routes say so.> fore, as it will *reroute* after masquerading. Bit of a hack, that. I > simply nixed the TOS bits in the firewalling code. :)The changes in 2.2 are a big hack. But the masquerading there will be not slower than the plain 2.2 kernel because the ip_route_output call is still permanently there to select maddr which is already known.> Summarizing: Yes, you can do equal cost multipath. Yes it is cool. Yes it > can be made nicer and friendlier to set up using Julian''s patches. However, > it will not be an ideal solution. Things *will* break. Load will just beThese patches fix small number of things. You can do more of the work (even the same work) with unpatched kernel and some scripts in user space. The problems come when the route cache entries expire.> approximately balanced. Failover is in most cases definitely not transparent > to the user: new connections have to be set up. If the links stay up though,The failover simply depends on gc_interval (used from the new connections only). As for the broken established connections: nobody can resurrect them if they have only one line available for the public IP addresses they use. If you can send traffic from one public IP through 2 external devices and if the 1st fails, the other will be used. Only the local connections bound to device will die, the NAT and the unbound TCP sockets can use another alive device if there are routes that say so.> equal cost multipath is a *good* thing. > > Oh, and it does work on >2 uplinks. I''ve set up a system for a client usingYes, it is tested, for example, with 2 WANs and one ethernet to route packets to another router with WAN. 2 routers, 3 WANs total. In short, solution for end hosts and routers that use border gateways. All other (VRRP, etc) solutions for failover are for intermediate routers. You can''t move the physical link from the failed box to another in some of the cases.> 1 ISDN line, 2 ADSL links (with the Dutch MXStream cruftiness, but I > digress) and 1 cable modem using masquerading only on the last three, using > the standard kernel (Julian''s patches didn''t exist a year ago, when I did > this). Worked splendidly (and still does, I''m told). Needed some manual > supervision though, as link failover and especially failback is *not* > trivial.There are some things in these patches that are not for the innocent Linux user: the need for active monitoring from user space for the gateways in the multipath and the alternative routes. I have an idea to do active monitoring in kernel but this is TODO. The problem is in fact for the multipath routes: we need valid neighbour state for these gateways (provided from ARP pings). If the ARP info is not known we can end with using only part of the paths in the multipath route and this bad, at least unexpected for the users that do not feed the kernel with fresh ARP info. But in such situation it seems they don''t demand failover detection.> Doei, Arthur.Regards -- Julian Anastasov <ja@ssi.bg>
On Tue, 4 Dec 2001 09:52:49 +0100 (MET) Arthur van Leeuwen <arthurvl@sci.kun.nl> wrote:> Okay, I''ve read both the nanohowto and the docs on Julian''s patches by now. > A few things to note: the nanohowto''s information is good even without > Julian''s patches, although things will become trickier. One has to do ones > own link-probing and rerouting from userland. That is very doable however, > provided you have machines somewhere at the ISP''s site that will answer to > either pings ore traceroutes or somesuch, as you will need answers.I''m not sure which is the situation Julian wanted to address with the patches, but having more than one line and using both of them at the same time, while being able to continue in case of failures, was mine to use those patches: I have no relation to my ISPs, they are not going to cooperate in any sense, often not even paying lots of money, and often they are doing things just to make it more difficult, because they want our money, but the do not want that we actually _use_ their installations (this is a special greating for spanish Telefónica and spanish Terra). If I had access to ISP cooperation, I could use a perfect line balancing between two known machines, or much easier: just ask for one bigger line, But the idea is just NOT to use the same ISP because they are unreliably. Again, just let me name as an example the spanish company Terra operating in Brazil, which has huge money and demonstrates record beating levels of incompetence and bad faith.> The patches Julian provided fix a bunch of nastiness. For one, dead gateway > detection is done on the ARP level in kernelspace. Very neat when you have > ARP, thus on ethernet, but not very useful without. Furthermore, they > provide true alternative routes, not only multipath default routes. This is > once more extremely neat, but not directly necessary for the usual case.This is not strictly true, as far as I understood Julians comments. The requirement is not to be ARP but to get some support from the link level, which is being the case, but which could be the case or at least could be done on other protocols as well. So if it doesn''t work with non ARP devices, the bug is not in Julians patches but in the implementation of that protocols.> Thirdly, Julian''s patches add gateways as a routing key. This will not help > pure routing boxes, such as would be standard issue in an office full of > Windows toasters, as the gateway will be determined at the routing stage, so > it cannot be used as a key.I did think of that, and one, at least indirect, consequence is that the gateways actually must be different. I had one case where both lines came from the same ISP (for regional reasons of availability) and I got two compatible IPs with the same gateway: IFE1 eth0 IPE1 200.201.202.28 GWE1 200.201.202.1 IFE2 eth1 IPE2 200.201.202.29 GWE1 200.201.202.1 (only the forth digits are the actual ones) Then I didn''t know of Julians patches and wrote a daemon which would reconfigure the network on failure or comeback of the lines. To do that, I used to have one explicit host route to each gateway, send pings to both and set the default route to the preferred line if both are working, or to the working line if one failed. With a patched or unpatched kernel, what would you do? To make it more difficult, this was still before 2.4 kernels, using route and ifconfig only, no ip(8). A priori, it is not obvious why this doesn''t work, but it just can''t work. I gave back one line and hired another one, because once again, the ISP didn''t want to cooperate, preferring to charge uninstallation and installation again (then some 300 US$). Not even arping broadcasts will work. Maybe a hack with source routed packets, but I usually disable this possibility in the kernel. Yes, the dependency on the gateway is a limitation, but it''s _much_ better than nothing. Maybe the solution would be to use more consequently the output device, but I guess this requires modifications which may reach the user interface. Julian''s packets didn''t do that, ip(8) & Co. is still used as before.> The *main* reason to use Julian''s patches is the masquerading connection > rerouting. This will fix the big bugs in your setup by just redirecting a > masqueraded connection out to a different interface when the old one is > dead.Could you elaborate which are those bugs in my setup? I would also be very grateful for any suggestion in fixing them.> This is *very* cool on UDP, and will make UDP failover to another > route fully transparent. However, it will not fix stateful protocols in > which the server on the other side keeps state on the IP address it was > talking to, such as SSH. It will fix the TOS nastiness OpenSSH brings to the > fore, as it will *reroute* after masquerading. Bit of a hack, that. I > simply nixed the TOS bits in the firewalling code. :)I also thought of a hack when I knew about the reroute at NAT time for masquerade, but then, first, it works, and second this seems to have been the way to not rewrite most of Linux networking stuff. Also, a stateful connection really can''t survive. We do not have any support at the ISP side; the connections are 100% independent, most of the times one ISP doesn''t (shouldn''t) even know of the other. There _is_ no solution to it. If you request a download, the remote server will start sending the packets to one particular IP. What are you suggesting to persuade that remote server, to continue sending them to another IP? I guess many crackers would love having such an oportunity to take over a connection! The remote server has no chance to know if the two IPs are in the same computer or thousands of kilometers apart.> Summarizing: Yes, you can do equal cost multipath. Yes it is cool. Yes it > can be made nicer and friendlier to set up using Julian''s patches. However, > it will not be an ideal solution. Things *will* break. Load will just be > approximately balanced. Failover is in most cases definitely not transparent > to the user: new connections have to be set up. If the links stay up though, > equal cost multipath is a *good* thing.No. Things do *not* break beyond having to restart certain connections (see above). I''m using this now since more than a week with hundreds of thousands of connections a day and it never broke. It works as smoothe as I was much to afraid to wish since a long time. The purpose of having more than one line is because the ISPs are unreliably. There are three to four line failures per day as an average. Nobody noticed ever any failover. OK, most of them are just surfing the net and not doing prolongated downloads or ssh session. I use ssh for maintainance and do know that a failing line can require reopening a new session. But now I can do that, and before I had to wait a couple of minutes until my daemon would reconfigure the network, including the netfilter rules, and establish the new situation. I did mention the result of load balancing in the nano howto; for now I observed a daily disbalance of maybe some 15%, which tend to diminish on longer periods of time.> Oh, and it does work on >2 uplinks. I''ve set up a system for a client using > 1 ISDN line, 2 ADSL links (with the Dutch MXStream cruftiness, but I > digress) and 1 cable modem using masquerading only on the last three, using > the standard kernel (Julian''s patches didn''t exist a year ago, when I did > this). Worked splendidly (and still does, I''m told). Needed some manual > supervision though, as link failover and especially failback is *not* > trivial.Well, if you did that, you did it very much in secret! Many persons around the globe, including myself, asked on this and other lists for an answer. I do remember a two-liner answer from you, which in that form was useless because it just wouldn''t work. A second direct question remained without reply. Maybe actually it didn''t work so splendidly after all, because after having studied Julians patches, I don''t really see how it could be done. But then, you dispose obviously of much more advance knowledge. -- Christoph Simon ciccio@kiosknet.com.br --- ^X^C q quit :q ^C end x exit ZZ ^D ? help .
... > > dead. This is *very* cool on UDP, and will make UDP failover to another > > route fully transparent. However, it will not fix stateful protocols in > > which the server on the other side keeps state on the IP address it was > > talking to, such as SSH. Sending packets with the same source address out another link only helps UDP if it''s not necessary to get replies, in which case I wouldn''t call it a "connection". If you didn''t need the replies then you could do the same thing for TCP. Note that this trick only works at all if your ISPs are not doing the ingress filtering they should - a safe assumption for now, but I hope not in the future. The problem is that you normally don''t have any control over the routing between two places. Even though your ISPs will let you send out packets with the source addresses belonging to each other, the replies will be routed to the one that "should" have sent the packets, and if that one is down you can''t get the replies. This problem could be fixed by extending TCP (and of course, changing the kernel) to support multiple IP addresses. I suggest a new option that says "here''s another IP address for me" (or perhaps, here''s an alternative IP/port). The kernel then has to merge these two input streams. On the output side (when you send to someone who has told you about alternative addresses) I can think of several ways to control which address you send to. I suppose the application should have some way to influence that, but as a first stab, I suggest that whenever tcp has to resend a packet, it should move to the next address. Anyone interested in trying to implement it? Put it on your queue and post to the list if you ever get to it. Also please post to the list any bugs (and fixes) you see in the proposal.
On Tue, 4 Dec 2001, Don Cohen wrote:> ... > > > dead. This is *very* cool on UDP, and will make UDP failover to another > > > route fully transparent. However, it will not fix stateful protocols in > > > which the server on the other side keeps state on the IP address it was > > > talking to, such as SSH. > > Sending packets with the same source address out another link only > helps UDP if it''s not necessary to get replies, in which case I > wouldn''t call it a "connection". If you didn''t need the replies then > you could do the same thing for TCP.Ah, but the rerouting patch doesn''t do that. It reroutes and *remasquerades* packets going out. You will only lose return packets destined to the other interface. The problem really only lies in the other end maintaining state in the higher level protocol.> This problem could be fixed by extending TCP (and of course, changing > the kernel) to support multiple IP addresses. I suggest a new option > that says "here''s another IP address for me" (or perhaps, here''s an > alternative IP/port). The kernel then has to merge these two input > streams. On the output side (when you send to someone who has told > you about alternative addresses) I can think of several ways to > control which address you send to. I suppose the application should > have some way to influence that, but as a first stab, I suggest that > whenever tcp has to resend a packet, it should move to the next > address.Ooh, that''d be cool. Building your own anycast group dynamically... and registering on the other side as said anycast group. Unfortunately, IPv4 doesn''t allow for IP anycasting. IPv6, anyone? :) Oh, you''re talking about implementing it at the TCP level? Right then... right. That should be possible... if only programs couldn''t bind to specific addresses... Doei, Arthur. (Note: the idea is one of the coolest I''ve seen in a while) -- /\ / | arthurvl@sci.kun.nl | Work like you don''t need the money /__\ / | A friend is someone with whom | Love like you have never been hurt / \/__ | you can dare to be yourself | Dance like there''s nobody watching
Arthur van Leeuwen writes: > > This problem could be fixed by extending TCP (and of course, changing > > the kernel) to support multiple IP addresses. I suggest a new option > > that says "here''s another IP address for me" (or perhaps, here''s an > > alternative IP/port). The kernel then has to merge these two input > > streams. On the output side (when you send to someone who has told > > you about alternative addresses) I can think of several ways to > > control which address you send to. I suppose the application should > > have some way to influence that, but as a first stab, I suggest that > > whenever tcp has to resend a packet, it should move to the next > > address. ... > Oh, you''re talking about implementing it at the TCP level? Right then... > right. That should be possible... if only programs couldn''t bind to specific > addresses... Right, I forgot about the other half, what interface programs will have to control their own alternative addresses. While I do think there should be such interfaces, I suggest a default that does the "right" thing for applications that don''t know about the extension. There should be some command to tell the kernel about the alternatives, e.g., whenever you allocate a port for address 1.2.3.* you should also allocate alternatives for addresses 6.7.8.9 and 4.5.6.7.