libvirt users - Oct 2012 - IPv4 routed virtual networks

OK, either I have taken a stupid pill and am missing something basic or 
routed network do not work.

I assume that, if they did work, it would be in more or less that same 
manner as a nat network as far as ping'ing, ssh'ing, etc. to another 
real host on the same real LAN as the virtualization host. At least that 
is what I believe I should expect.

I have googled for info and everything says that it should work.  On of 
the referenced documents was this: 
http://berrange.com/posts/2009/12/13/routed-subnets-without-nat-for-libvirt-managed-virtual-machines-in-fedora/

I checked through everything and it all matches what is in the 
document.  I can go from the guest to the host (ping, ssh, etc) but not 
from that guest to another real host on the same real LAN.

I am patching the relevant info below:

----------------------------------
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.p32p1.forwarding = 1
net.ipv4.conf.virbr8.forwarding = 1
--------------------------------
<network>
   <name>routed</name>
   <uuid>1b2a0197-e708-165c-f266-6822e73cfbdd</uuid>
   <forward dev='p32p1' mode='route'>
     <interface dev='p32p1'/>
   </forward>
   <bridge name='virbr8' stp='on' delay='0' />
   <mac address='52:54:00:B9:59:49'/>
   <domain name='routed'/>
   <ip address='192.168.123.1' netmask='255.255.255.0'>
     <dhcp>
       <range start='192.168.123.128' end='192.168.123.254'
/>
     </dhcp>
   </ip>
</network>
-------------------------------
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source destination
     0     0 ACCEPT     udp  --  virbr8 *       0.0.0.0/0 
0.0.0.0/0            udp dpt:53
     0     0 ACCEPT     tcp  --  virbr8 *       0.0.0.0/0 
0.0.0.0/0            tcp dpt:53
     0     0 ACCEPT     udp  --  virbr8 *       0.0.0.0/0 
0.0.0.0/0            udp dpt:67
     0     0 ACCEPT     tcp  --  virbr8 *       0.0.0.0/0 
0.0.0.0/0            tcp dpt:67


Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source destination
     0     0 ACCEPT     all  --  p32p1  virbr8  0.0.0.0/0 192.168.123.0/24
     0     0 ACCEPT     all  --  virbr8 p32p1   192.168.123.0/24 0.0.0.0/0
     0     0 ACCEPT     all  --  virbr8 virbr8  0.0.0.0/0 0.0.0.0/0
     0     0 REJECT     all  --  *      virbr8  0.0.0.0/0 
0.0.0.0/0            reject-with icmp-port-unreachable
     0     0 REJECT     all  --  virbr8 *       0.0.0.0/0 
0.0.0.0/0            reject-with icmp-port-unreachable

------------------------------

This is Fedora 17 with an updated libvirt 0.10.1-4.fc17 [another system 
with the same "problem" is running libvirt 0.10.2-1.fc17].

Anyone know what is going on?

BTW, I searched bugzilla for the comment containing the string routed 
filed against packages libvirt, qemu, qemu-kvm, or kvm ...Result .. no hits!

If this is really a bug rather than something I did or did not do, then 
nobody is using routed virtual networks.

BTW, I have a real F17 systems sitting between to networks on two 
different NICs and it routes things nicely, thank you very much.  It 
does work but not for the virtual networks.

I would really like it to be something I am missing.

Gene

On 10/03/2012 10:08 PM, Gene Czarcinski wrote:
> OK, either I have taken a stupid pill and am missing something basic
> or routed network do not work.
>
> I assume that, if they did work, it would be in more or less that same
> manner as a nat network as far as ping'ing, ssh'ing, etc. to
another
> real host on the same real LAN as the virtualization host. At least
> that is what I believe I should expect.
I use routed networks, but they require some extra work. The docs [1]
say that "This is a variant on the default network which routes traffic
from the virtual network to the LAN _without applying any NAT_. It
requires that the IP address range be _pre-configured in the routing
tables of the router on the host network_." (emphasis mine) The way I
read into this is that you are responsible for routing packets yourself.
The docs say that you have to use IP addresses that are visible by the
host and external machines and configure your routing table, so that it
forwards packets to the VMs. I am not sure how to do that, but I have a
different solution -- I use a pair of IP addresses for each VM: external
(real and configured on the host) and internal. Then I have a SNAT +
DNAT pair of iptables rules that expose the VM to the outside world in a
manner similar to the one used in DMZ setups:

ubuntu at thehost:~$ sudo iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination        
DNAT       all  --  0.0.0.0/0            X.Y.Z.120           to:10.1.1.2

...

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination        
SNAT       all  --  10.1.1.2             0.0.0.0/0            to:X.Y.Z.120

I am not sure if anyone else is doing the SNAT/DNAT thing to routed
networks, but it's been working quite nicely for me so far.

Best,
Hristo

[1] http://libvirt.org/formatnetwork.html#examplesRoute
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20121003/b4b34a95/attachment.htm>