John A. Sullivan III
2005-Sep-17 18:29 UTC
[Xen-users] Xen IP stack fails handoff to pluto in openswan
This one has me stumped. I have openswan 2.3.0 installed on a xen 2.0.7
virtual machine running fedora core 3. I''ve tried establishing tunnels
with a CyberGuard SG575 (FreeS/WAN), an old Super-FreeS/WAN gatway and a
Windows IPSec only client (lsipsectool -
http://sf.net/projects/lsipsectool). I see the same symptom in all
cases so I suspect the problem is between openswan and xen. Pluto never
see packets destined for it.
I''ve looked at this several ways. I''ll refer to the Xen
openswan
gateway as XenOSW. tcpdump on XenOSW sees that packets on eth0. If I
log the packets on the INPUT chain of iptables on XenOSW, I see them
there, too. I set plutodebug=all in ipsec.conf but I still do not see
any replies or initiations from the partner even though I see them on
the OUTPUT and INPUT chains and the eth0 interface.
In /var/log/secure I get plenty of:
Sep 17 13:31:14 NiagaraRASGW pluto[604]: | emitting length of ISAKMP
Vendor ID Payload: 20
Sep 17 13:31:14 NiagaraRASGW pluto[604]: | emitting length of ISAKMP
Message: 292
Sep 17 13:31:14 NiagaraRASGW pluto[604]: | sending 292 bytes for
main_outI1 through eth0:500 to x.x.x.188:500:
but never a reply and I never see any packet received messages from any
of the partners even though we see the packets on the interface.
Here are the INPUT chain iptables rules (which work perfectly on non Xen
openswan gateways):
Chain INPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
8 560 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 state RELATED,ESTABLISHED
0 0 ACCEPT tcp -- * * 0.0.0.0/0
0.0.0.0/0 tcp dpt:22 state NEW
0 0 ACCEPT tcp -- lo * 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT esp -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT udp -- * * 0.0.0.0/0
0.0.0.0/0 udp dpt:500
0 0 ACCEPT udp -- * * 0.0.0.0/0
0.0.0.0/0 udp dpt:4500
0 0 VPN_ALLOW all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCESS_GROUPS_DENY all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCESS_GROUPS all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 LOG all -- * * 0.0.0.0/0
0.0.0.0/0 LOG flags 0 level 4 prefix `No Match: ''
Here is OUTPUT:
Chain OUTPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
30 5888 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 state RELATED,ESTABLISHED
0 0 ACCEPT udp -- * * 0.0.0.0/0
0.0.0.0/0 udp dpt:53 state NEW
0 0 ACCEPT udp -- * * 0.0.0.0/0
0.0.0.0/0 udp dpt:123 state NEW
0 0 ACCEPT tcp -- * * 0.0.0.0/0
0.0.0.0/0 tcp dpt:22 state NEW
0 0 ACCEPT tcp -- * lo 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT esp -- * * 0.0.0.0/0
0.0.0.0/0
3 960 ACCEPT udp -- * * 0.0.0.0/0
0.0.0.0/0 udp dpt:500
0 0 ACCEPT udp -- * * 0.0.0.0/0
0.0.0.0/0 udp dpt:4500
0 0 VPN_ALLOW all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCESS_GROUPS_DENY all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCESS_GROUPS all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 LOG all -- * * 0.0.0.0/0
0.0.0.0/0 LOG flags 0 level 4 prefix `No Match: ''
The Xen host has two NICs. All guests except the XenOSW use eth1 on
bridge xen-br0.
The XenOSW domU uses eth0 through bridge xen-br1 and has a manually
defined MAC address of 02:00:00:00:00:02. There is no IP address bound
to eth0 or xen-br1 in dom0 (the host). The IP address is bound in
XenOSW. We do this because we do not want to expose the dom0 to the
Internet in any way. However, we have tried it with a legitimate
address bound to the host eth0 and to bridge xen-br1.
The XenOSW domU does not start automatically as it is still a test
device. Instead, after the dom0 boots, we do:
brctl addbr xen-br1
brctl addif xen-br1 eth0
ifconfig xen-br1 up
We then boot the XenOSW domU and all other traffic seems fine, e.g., the
iptables list was taken from an SSH session between my laptop and the
XenOSW. Just Pluto is broken.
I have no idea what is wrong or even how to troubleshoot it. The
packets just seem to fail on the handoff from the IP stack to the Pluto
application. Any suggestions about either what is wrong or how to
troubleshoot it further? Many thanks to anyone willing to dive in this
deep! - John
--
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@opensourcedevel.com
If you would like to participate in the development of an open source
enterprise class network security management system, please visit
http://iscs.sourceforge.net
--
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@opensourcedevel.com
Financially sustainable open source development
http://www.opensourcedevel.com
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Hi - I''m just experimenting with XEN and so far I am very impressed :) But I''m wondering about domain0.... Should I treat domain0 as a slimmed down management domain which I just use to run the other domU''s or can I use domain0 for some major services which the machine is running (web,smtp etc)? What''s best practice? I''d like to minimise the number of IP''s I need for my XEN box, so ideally would like to use dom0 for serveral server functions (in addition to running XEN). Thanks, Paul _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
John A. Sullivan III
2005-Sep-19 10:07 UTC
Re: [Xen-users] Xen IP stack fails handoff to pluto in openswan
On Sun, 2005-09-18 at 07:49 -0400, Ted Kaczmarek wrote:> On Sat, 2005-09-17 at 14:29 -0400, John A. Sullivan III wrote: > > This one has me stumped. I have openswan 2.3.0 installed on a xen 2.0.7 > > virtual machine running fedora core 3. I''ve tried establishing tunnels > > with a CyberGuard SG575 (FreeS/WAN), an old Super-FreeS/WAN gatway and a > > Windows IPSec only client (lsipsectool - > > http://sf.net/projects/lsipsectool). I see the same symptom in all > > cases so I suspect the problem is between openswan and xen. Pluto never > > see packets destined for it. > > > > I''ve looked at this several ways. I''ll refer to the Xen openswan > > gateway as XenOSW. tcpdump on XenOSW sees that packets on eth0. If I > > log the packets on the INPUT chain of iptables on XenOSW, I see them > > there, too. I set plutodebug=all in ipsec.conf but I still do not see > > any replies or initiations from the partner even though I see them on > > the OUTPUT and INPUT chains and the eth0 interface. > > > > In /var/log/secure I get plenty of: > > > > Sep 17 13:31:14 NiagaraRASGW pluto[604]: | emitting length of ISAKMP > > Vendor ID Payload: 20 > > Sep 17 13:31:14 NiagaraRASGW pluto[604]: | emitting length of ISAKMP > > Message: 292 > > Sep 17 13:31:14 NiagaraRASGW pluto[604]: | sending 292 bytes for > > main_outI1 through eth0:500 to x.x.x.188:500: > > > > but never a reply and I never see any packet received messages from any > > of the partners even though we see the packets on the interface. > > > > Here are the INPUT chain iptables rules (which work perfectly on non Xen > > openswan gateways): > > Chain INPUT (policy DROP 0 packets, 0 bytes) > > pkts bytes target prot opt in out source > > destination > > 8 560 ACCEPT all -- * * 0.0.0.0/0 > > 0.0.0.0/0 state RELATED,ESTABLISHED > > 0 0 ACCEPT tcp -- * * 0.0.0.0/0 > > 0.0.0.0/0 tcp dpt:22 state NEW > > 0 0 ACCEPT tcp -- lo * 0.0.0.0/0 > > 0.0.0.0/0 > > 0 0 ACCEPT esp -- * * 0.0.0.0/0 > > 0.0.0.0/0 > > 0 0 ACCEPT udp -- * * 0.0.0.0/0 > > 0.0.0.0/0 udp dpt:500 > > 0 0 ACCEPT udp -- * * 0.0.0.0/0 > > 0.0.0.0/0 udp dpt:4500 > > 0 0 VPN_ALLOW all -- * * 0.0.0.0/0 > > 0.0.0.0/0 > > 0 0 ACCESS_GROUPS_DENY all -- * * 0.0.0.0/0 > > 0.0.0.0/0 > > 0 0 ACCESS_GROUPS all -- * * 0.0.0.0/0 > > 0.0.0.0/0 > > 0 0 LOG all -- * * 0.0.0.0/0 > > 0.0.0.0/0 LOG flags 0 level 4 prefix `No Match: '' > > > > Here is OUTPUT: > > Chain OUTPUT (policy DROP 0 packets, 0 bytes) > > pkts bytes target prot opt in out source > > destination > > 30 5888 ACCEPT all -- * * 0.0.0.0/0 > > 0.0.0.0/0 state RELATED,ESTABLISHED > > 0 0 ACCEPT udp -- * * 0.0.0.0/0 > > 0.0.0.0/0 udp dpt:53 state NEW > > 0 0 ACCEPT udp -- * * 0.0.0.0/0 > > 0.0.0.0/0 udp dpt:123 state NEW > > 0 0 ACCEPT tcp -- * * 0.0.0.0/0 > > 0.0.0.0/0 tcp dpt:22 state NEW > > 0 0 ACCEPT tcp -- * lo 0.0.0.0/0 > > 0.0.0.0/0 > > 0 0 ACCEPT esp -- * * 0.0.0.0/0 > > 0.0.0.0/0 > > 3 960 ACCEPT udp -- * * 0.0.0.0/0 > > 0.0.0.0/0 udp dpt:500 > > 0 0 ACCEPT udp -- * * 0.0.0.0/0 > > 0.0.0.0/0 udp dpt:4500 > > 0 0 VPN_ALLOW all -- * * 0.0.0.0/0 > > 0.0.0.0/0 > > 0 0 ACCESS_GROUPS_DENY all -- * * 0.0.0.0/0 > > 0.0.0.0/0 > > 0 0 ACCESS_GROUPS all -- * * 0.0.0.0/0 > > 0.0.0.0/0 > > 0 0 LOG all -- * * 0.0.0.0/0 > > 0.0.0.0/0 LOG flags 0 level 4 prefix `No Match: '' > > > > The Xen host has two NICs. All guests except the XenOSW use eth1 on > > bridge xen-br0. > > > > The XenOSW domU uses eth0 through bridge xen-br1 and has a manually > > defined MAC address of 02:00:00:00:00:02. There is no IP address bound > > to eth0 or xen-br1 in dom0 (the host). The IP address is bound in > > XenOSW. We do this because we do not want to expose the dom0 to the > > Internet in any way. However, we have tried it with a legitimate > > address bound to the host eth0 and to bridge xen-br1. > > > > The XenOSW domU does not start automatically as it is still a test > > device. Instead, after the dom0 boots, we do: > > brctl addbr xen-br1 > > brctl addif xen-br1 eth0 > > ifconfig xen-br1 up > > > > We then boot the XenOSW domU and all other traffic seems fine, e.g., the > > iptables list was taken from an SSH session between my laptop and the > > XenOSW. Just Pluto is broken. > > > > I have no idea what is wrong or even how to troubleshoot it. The > > packets just seem to fail on the handoff from the IP stack to the Pluto > > application. Any suggestions about either what is wrong or how to > > troubleshoot it further? Many thanks to anyone willing to dive in this > > deep! - John > > -- > > John A. Sullivan III > > Open Source Development Corporation > > +1 207-985-7880 > > jsullivan@opensourcedevel.com > > > This really doesn''t appear to be Xen related so direct reply seemed more > appropriate. > > Do you have forward rules to match the subnet declarations for your > tunnels? > > Don''t see any in the above rules, not sure if this applies with klips or > not, I really miss the old ipsec0 type interface. > > ipsec barf can be most helpful also. > > > Regards, > Ted > > >Thanks. We are using KLIPS on a 2.4 kernel (the host is a 2.6 kernel) and thus have ipsec interfaces. There are proper FORWARD rules. I didn''t include them because we never get to an IPSec SA. We never get past the very first IKE packet. Nothing shows up in the pluto logs. There is no sign that pluto ever received the packet from the IP stack. That is why I am suspecting that it might possibly be a Xen problem. There are also no errors in the xend logs. The vif comes up fine and there are not errors. Thus my great confusion! By the way, we are using bridging mode and not routing mode. -- John A. Sullivan III Open Source Development Corporation +1 207-985-7880 jsullivan@opensourcedevel.com If you would like to participate in the development of an open source enterprise class network security management system, please visit http://iscs.sourceforge.net _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> But I''m wondering about domain0.... Should I treat domain0 > as a slimmed down management domain which I just use to run > the other domU''s or can I use domain0 for some major services > which the machine is running (web,smtp etc)?It''s more secure to keep dom0 minimal. Also, for most services, there''s no need to run them in dom0: they should run just as fast in a domU, with better isolation.> What''s best practice? I''d like to minimise the number of IP''s I need > for my XEN box, so ideally would like to use dom0 for serveral > server functions (in addition to running XEN).How about configuring Iptables rules to redirect the appropriate ports to different domains? You won''t need loads of IP addresses but you''ll get the proper Xen isolation. Cheers, Mark _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users