I have two physical hosts (Correo and Alexander), running two XEN
instances on one of them (Ferrari and Amarillo on Correo) and one
on the other (Graham on Alexander)...
Picture at http://bayour.com/misc/VoIP.jpg.
On the firewall/gateway (192.168.1.1) I route 192.168.3.0/24 to Correo
(192.168.1.7) and 192.168.4.0/24 to Alexander (192.168.1.6). This so
that I can access the XEN hosts from the internal network. Very basic...
And all my VoIP phones is on it''s (about to be on a) separate network
with the firewall/gateway as default gateway.
On Alexander:
============ * /etc/xen/graham.cfg
kernel = ''/boot/vmlinuz-2.6.18-5-xen-amd64''
ramdisk = ''/boot/initrd.img-2.6.18-5-xen-amd64''
memory = ''2500''
root = ''/dev/sda1 ro''
disk = [
''file:/home/xen/domains/graham/disk.img,sda1,w'',
''file:/home/xen/domains/graham/swap.img,sda2,w'' ]
name = ''graham''
vif = [ ''ip=192.168.4.11'' ]
on_poweroff = ''destroy''
on_reboot = ''restart''
on_crash = ''restart''
* /etc/xen/xend-config.sxp
(xend-http-server yes)
(xend-unix-server yes)
(xend-tcp-xmlrpc-server no)
(xend-unix-xmlrpc-server yes)
(xend-relocation-server yes)
(xend-unix-path /var/lib/xend/xend-socket)
(xend-port 8000)
(xend-relocation-port 8002)
(xend-address ''alexander'')
(xend-relocation-address ''alexander'')
(console-limit 1024)
(network-script network-route)
(vif-script vif-route)
(dom0-min-mem 196)
(dom0-cpus 2)
(enable-dump yes)
(vnc-listen ''0.0.0.0'')
* ifconfig (trimmed - only ''lo'' if removed)
eth0 Link encap:Ethernet HWaddr 00:1C:23:C4:28:92
inet addr:192.168.1.6 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::21c:23ff:fec4:2892/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth0:0 Link encap:Ethernet HWaddr 00:1C:23:C4:28:92
inet addr:192.168.4.1 Bcast:192.168.4.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
vif5.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet addr:192.168.1.6 Bcast:192.168.1.255 Mask:255.255.255.255
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
* route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
192.168.4.11 0.0.0.0 255.255.255.255 UH 0 0 0
vif5.0
192.168.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth0
* iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT 0 -- 192.168.4.11 0.0.0.0/0 PHYSDEV match
--physdev-in vif5.0
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match
--physdev-in vif5.0 udp spt:68 dpt:67
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
* iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
On Graham:
========= * ifconfig (trimmed - only ''lo'' if removed)
eth0 Link encap:Ethernet HWaddr 00:16:3E:00:AB:28
inet addr:192.168.4.11 Bcast:192.168.4.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
* route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
192.168.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
0.0.0.0 192.168.4.1 0.0.0.0 UG 0 0 0 eth0
* iptables -L -n
FATAL: Could not load /lib/modules/2.6.18-5-xen-amd64/modules.dep: No such
file or directory
iptables v1.3.6: can''t initialize iptables table `filter'':
iptables who? (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
Correo with the XEN hosts Ferrari and Amarillo basically look identical (only
different
networks).
As seen, I do NOT use NAT here. I wanted to use true routed network... And it
seems to work.
My primary Asterisk server (the one that do all the routing - the one on
Alexander only deals
with the PSTN trafik) runs on Graham and it can be accessed from the outside -
with port
forwarding on the firewall/gateway and it can also contact external Asterisk
servers (I run
one at home to deal with my private VoIP).
The DNS runs on Correo, but it can not be reached (queried) from Graham!
----- s n i p -----
graham# ping -c 5 correo
ping: unknown host correo
graham# ping -c 5 192.168.1.7
PING 192.168.1.7 (192.168.1.7) 56(84) bytes of data.
64 bytes from 192.168.1.7: icmp_seq=1 ttl=62 time=0.270 ms
64 bytes from 192.168.1.7: icmp_seq=2 ttl=62 time=0.260 ms
64 bytes from 192.168.1.7: icmp_seq=3 ttl=62 time=0.264 ms
64 bytes from 192.168.1.7: icmp_seq=4 ttl=62 time=0.273 ms
64 bytes from 192.168.1.7: icmp_seq=5 ttl=62 time=0.257 ms
--- 192.168.1.7 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4000ms
rtt min/avg/max/mdev = 0.257/0.264/0.273/0.021 ms
graham# traceroute -n 192.168.1.7
traceroute to 192.168.1.7 (192.168.1.7), 30 hops max, 52 byte packets
1 192.168.1.6 0.285 ms 0.091 ms 0.090 ms
2 192.168.1.7 0.323 ms 0.262 ms 0.258 ms
graham# telnet 192.168.1.7 53
Trying 192.168.1.7...
Connected to 192.168.1.7.
Escape character is ''^]''.
correo
Connection closed by foreign host.
graham# host graham 192.168.1.7
;; reply from unexpected source: 192.168.1.1#53, expected 192.168.1.7#53
;; reply from unexpected source: 192.168.1.1#53, expected 192.168.1.7#53
;; connection timed out; no servers could be reached
----- s n i p -----
Also, scp or ssh FROM Graham to Correo don''t work, but the other way
around works fine...
Looking at the answer that ''host'' gave me, I now see that the
connection
goes via the firewall/gateway which is not directly obvious - Alexander
(which is Graham''s default GW) is on the same network as Correo...
PS. I solved this specific DNS problem with a caching DNS server on
Alexander, but scp/ssh (etc) naturally still don''t work because
of this weird problem... I just can''t see it! Maybe a set of
(many :) extra eyes can... Thanx!
On Thu, 2008-04-10 at 14:55 +0200, Turbo Fredriksson wrote:> I have two physical hosts (Correo and Alexander), running two XEN > instances on one of them (Ferrari and Amarillo on Correo) and one > on the other (Graham on Alexander)... > > Picture at http://bayour.com/misc/VoIP.jpg. > > > On the firewall/gateway (192.168.1.1) I route 192.168.3.0/24 to Correo > (192.168.1.7) and 192.168.4.0/24 to Alexander (192.168.1.6). This so > that I can access the XEN hosts from the internal network. Very basic... > > And all my VoIP phones is on it''s (about to be on a) separate network > with the firewall/gateway as default gateway. > > > On Alexander: > ============> * /etc/xen/graham.cfg > kernel = ''/boot/vmlinuz-2.6.18-5-xen-amd64'' > ramdisk = ''/boot/initrd.img-2.6.18-5-xen-amd64'' > memory = ''2500'' > root = ''/dev/sda1 ro'' > disk = [ ''file:/home/xen/domains/graham/disk.img,sda1,w'', ''file:/home/xen/domains/graham/swap.img,sda2,w'' ] > name = ''graham'' > vif = [ ''ip=192.168.4.11'' ] > on_poweroff = ''destroy'' > on_reboot = ''restart'' > on_crash = ''restart'' > > * /etc/xen/xend-config.sxp > (xend-http-server yes) > (xend-unix-server yes) > (xend-tcp-xmlrpc-server no) > (xend-unix-xmlrpc-server yes) > (xend-relocation-server yes) > (xend-unix-path /var/lib/xend/xend-socket) > (xend-port 8000) > (xend-relocation-port 8002) > (xend-address ''alexander'') > (xend-relocation-address ''alexander'') > (console-limit 1024) > (network-script network-route) > (vif-script vif-route) > (dom0-min-mem 196) > (dom0-cpus 2) > (enable-dump yes) > (vnc-listen ''0.0.0.0'') > > * ifconfig (trimmed - only ''lo'' if removed) > eth0 Link encap:Ethernet HWaddr 00:1C:23:C4:28:92 > inet addr:192.168.1.6 Bcast:192.168.1.255 Mask:255.255.255.0 > inet6 addr: fe80::21c:23ff:fec4:2892/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > eth0:0 Link encap:Ethernet HWaddr 00:1C:23:C4:28:92 > inet addr:192.168.4.1 Bcast:192.168.4.255 Mask:255.255.255.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > vif5.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF > inet addr:192.168.1.6 Bcast:192.168.1.255 Mask:255.255.255.255 > inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > * route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use Iface > 192.168.4.11 0.0.0.0 255.255.255.255 UH 0 0 0 vif5.0 > 192.168.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 > 0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth0 > > * iptables -L -n > Chain INPUT (policy ACCEPT) > target prot opt source destination > > Chain FORWARD (policy ACCEPT) > target prot opt source destination > ACCEPT 0 -- 192.168.4.11 0.0.0.0/0 PHYSDEV match --physdev-in vif5.0 > ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in vif5.0 udp spt:68 dpt:67 > > Chain OUTPUT (policy ACCEPT) > target prot opt source destination > > * iptables -L -n -t nat > Chain PREROUTING (policy ACCEPT) > target prot opt source destination > > Chain POSTROUTING (policy ACCEPT) > target prot opt source destination > > Chain OUTPUT (policy ACCEPT) > target prot opt source destination > > On Graham: > =========> * ifconfig (trimmed - only ''lo'' if removed) > eth0 Link encap:Ethernet HWaddr 00:16:3E:00:AB:28 > inet addr:192.168.4.11 Bcast:192.168.4.255 Mask:255.255.255.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > * route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use Iface > 192.168.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 > 0.0.0.0 192.168.4.1 0.0.0.0 UG 0 0 0 eth0 > > * iptables -L -n > FATAL: Could not load /lib/modules/2.6.18-5-xen-amd64/modules.dep: No such file or directory > iptables v1.3.6: can''t initialize iptables table `filter'': iptables who? (do you need to insmod?) > Perhaps iptables or your kernel needs to be upgraded. > > Correo with the XEN hosts Ferrari and Amarillo basically look identical (only different > networks). > > As seen, I do NOT use NAT here. I wanted to use true routed network... And it seems to work. > My primary Asterisk server (the one that do all the routing - the one on Alexander only deals > with the PSTN trafik) runs on Graham and it can be accessed from the outside - with port > forwarding on the firewall/gateway and it can also contact external Asterisk servers (I run > one at home to deal with my private VoIP). > > > The DNS runs on Correo, but it can not be reached (queried) from Graham! > > ----- s n i p ----- > graham# ping -c 5 correo > ping: unknown host correo > > graham# ping -c 5 192.168.1.7 > PING 192.168.1.7 (192.168.1.7) 56(84) bytes of data. > 64 bytes from 192.168.1.7: icmp_seq=1 ttl=62 time=0.270 ms > 64 bytes from 192.168.1.7: icmp_seq=2 ttl=62 time=0.260 ms > 64 bytes from 192.168.1.7: icmp_seq=3 ttl=62 time=0.264 ms > 64 bytes from 192.168.1.7: icmp_seq=4 ttl=62 time=0.273 ms > 64 bytes from 192.168.1.7: icmp_seq=5 ttl=62 time=0.257 ms > > --- 192.168.1.7 ping statistics --- > 5 packets transmitted, 5 received, 0% packet loss, time 4000ms > rtt min/avg/max/mdev = 0.257/0.264/0.273/0.021 ms > > graham# traceroute -n 192.168.1.7 > traceroute to 192.168.1.7 (192.168.1.7), 30 hops max, 52 byte packets > 1 192.168.1.6 0.285 ms 0.091 ms 0.090 ms > 2 192.168.1.7 0.323 ms 0.262 ms 0.258 ms > > graham# telnet 192.168.1.7 53 > Trying 192.168.1.7... > Connected to 192.168.1.7. > Escape character is ''^]''. > correo > Connection closed by foreign host. > > graham# host graham 192.168.1.7 > ;; reply from unexpected source: 192.168.1.1#53, expected 192.168.1.7#53 > ;; reply from unexpected source: 192.168.1.1#53, expected 192.168.1.7#53 > ;; connection timed out; no servers could be reached > ----- s n i p ----- > > Also, scp or ssh FROM Graham to Correo don''t work, but the other way > around works fine... > > > Looking at the answer that ''host'' gave me, I now see that the connection > goes via the firewall/gateway which is not directly obvious - Alexander > (which is Graham''s default GW) is on the same network as Correo... > > > PS. I solved this specific DNS problem with a caching DNS server on > Alexander, but scp/ssh (etc) naturally still don''t work because > of this weird problem... I just can''t see it! Maybe a set of > (many :) extra eyes can... Thanx!Did you figure it out yet? I can not quite tell what you''re doing. Who is that blue router in your diagram? Maybe you are using a router icon to indicate a switch? At any rate if I had to guess it sounds like you are expecting to speak from eth0:0 IP when you are actually speaking from eth0 IP. This you could problably confirm with a tcpdump on the target machine while you probe from the other. If so, you could fix that with policy routing AKA source routing. i.e. `ip rule help`. Look at your ARP tables too just in case.
Quoting Dale Bewley <dlbewley@lib.ucdavis.edu>:> Did you figure it out yet?No I didn''t.> I can not quite tell what you''re doing. Who is that blue router in your > diagram? Maybe you are using a router icon to indicate a switch?I am, sorry. Didn''t find any switch icons in dia.> At any rate if I had to guess it sounds like you are expecting to speak > from eth0:0 IP when you are actually speaking from eth0 IP. This you > could problably confirm with a tcpdump on the target machine while you > probe from the other. If so, you could fix that with policy routing AKA > source routing. i.e. `ip rule help`. Look at your ARP tables too just in > case.I''ll have a look. Thanx for the idea...