Xen users - Sep 2007 - Port forwarding from dom0 to bridged domU with IPVS

Hi list,

We, an OSS advocacy group, setup a Xen 3.1 machine composed of :

. a 64 bits dom0 running Debian stable amd64
. 2 hvm domUs running OpenBSD amd64
. 2 hvm domUs running NetBSD i386

This machine is to be hosted and reachable from the Internet, but it will 
only have one public IP.
Naturally, our first tought was to port-forward using iptables / 
netfilter.  We didn''t really though it would be an issue... and that 
was a mistake :)
We tried many options, read many hints, even on this list, but no matter 
what, the port-forwarding, using a ultra-classic PREROUTING / FORWARD 
rule, was given a TCP RST in the best scenario. We read here stories about 
activating NAT / masquerading on the domU to fix (???) this issue, but as 
the machine is meant to be hosted, that was not the cleanest approach.

And then we took a look at IPVS 
(http://www.linuxvirtualserver.org/software/ipvs.html), an opensource 
Linux kernel module initially meant to act as a loadbalancer. We thought 
that providing a unique real server (the domU) to the VIP would do the 
trick... and it did ! Here''s a quick example of a working configuration
:

. dom0 has a public IP address, no services but ssh available
. domU has a RFC1918 address, linked to a bridge on the second ethernet
   interface of the dom0

We want to redirect the port 2222 of the dom0 to the port 22 of the domU :

- Install ipvsadm on the dom0 (apt-get install ipvsadm on debian)
- Setup the VIP :

# ipvsadm -A -t <public_ip>:2222 -s rr

We choosed the Round-Robin algorithm, but obviously this has no effect for 
us as there will be only one real server behind the loadbalancer

- Insert domU''s private IP on the VIP :

# ipvsadm -a -t <public_ip>:2222 -r <domU_private_ip>:22 -m

Here we use the simple masquerading mode of IPVS

- See the output :

root@dom0:~# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
   -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  dom0:2222 rr
   -> shells:ssh                   Masq    1      0          0

And finally, from an outside machine :

imil@tatooine:~$ ssh -p 2222 dom0
imil@dom0_public_ip''s password:
Last login: Sun Sep 16 01:15:40 2007 from somewhere_else
OpenBSD 4.1 (GENERIC) #874: Sat Mar 10 19:09:51 MST 2007

imil@shells
~$

It Works !

Hope this method can save time to some of you, for us it''s now the
perfect
solution as it provides us also the ability to loadbalance services on 
other domU''s.

Regards,

----------------------------------------
Emile "iMil" Heitor <imil@home.imil.net>                        
_
                    http://gcu-squad.org            ASCII ribbon campaign ( )
                                                     - against HTML email  X
                                                                 & vCards /
\

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users