I''m still trying to get Shorewall working properly on a new Xen system. My current problem is that SNAT doesn''t seem to work. When I connect out from behind my new firewall to another host, the source IP address is supposed to be SNAT''ed to the address of my firewall, but this isn''t happening. The outbound connection =does= succeed -- just not with the desired masquerading of the source IP address. I went back to Shorewall 3.4.4, in case there might be some bug in the 4.0.6 code I had been using, but the problem is unchanged in 3.4.4. The attached diagnostic output is from 3.4.4. I''m attaching the output of "shorewall dump". I''m also attaching a TAR file with my configuration (based on the "two-interfaces" example) -- I tried to make this example as small as possible while still illustrating the problem. The network "behind" my new firewall is 172.31.53.0/24. The exterior network is 172.29.0.0/24 (which is also my home LAN''s network -- again, this system is still in development, so I''m working entirely within my home LAN for now). The experiment I did just before doing "shorewall dump" was to SSH from a host behind my firewall (172.31.53.5) to a host on my LAN (172.29.0.29). The SSH succeeded, but the source IP address was unchanged. I had hoped/ expected the connection to appear to have originated from my firewall''s exterior address (172.29.0.53) -- but the target host saw the source''s real address (172.31.53.5) instead. In the "NAT Table" portion of the "shorewall dump" output, you''ll see a chain called "lan0_masq", which I assume is supposed to change a source address in the 172.31.53.0/24 range to 172.29.0.53. Note that lan0_masq is invoked once in the POSTROUTING chain, but the lan0_masq chain doesn''t appear to be processing any packets. In case it may be helpful, here is a line from my syslog file (I added logging for SSH through the firewall). Note that the connection appears to have gone from 172.31.53.5 (my "dmz" network) to 172.29.0.29 (my "lan" network), as expected: Dec 26 20:07:23 whodunit kernel: [72660.851790] Shorewall:dmz2lan:ACCEPT:ssh IN=dmz0 OUT=lan0 SRC=172.31.53.5 DST=172.29.0.29 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=4635 DF PROTO=TCP SPT=57013 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0 My firewall is the dom0 of a routed Xen 3.1 / Ubuntu 7.10 system (kernel 2.6.22-14-xen). The network behind the firewall (172.31.53.0/24) holds my domU''s, though it also has a physical interface card (which currently doesn''t have anything plugged into it). The external network interface (171.66.155.243) is currently unconnected while I develop/test. Any suggestions as to how I can get SNAT working would be gratefully appreciated. Thanks. -- Rich Wales === Palo Alto, CA, USA === richw@richw.org http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Rich Wales wrote:> Any suggestions as to how I can get SNAT working would be gratefully > appreciated. Thanks.The IP configuration on this box looks really messed up -- a combination of Xen routed and Xen bridged configurations. I had understood from previous posts that you were attempting to establish a completely routed environment -- is that correct? If so, I would clean up the Xen configuration first so that you know the paths that packets are taking through the box. In a routed configuration, "brctl show" should not list any ports on the bridge xenbr0. I suspect that lan0 output is actually going through the bridge which will cause SNAT on lan0 to fail. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Tom Eastep wrote:> The IP configuration on this box looks really messed up -- a combination > of Xen routed and Xen bridged configurations. . . . I would clean up > the Xen configuration first so that you know the paths that packets are > taking through the box.Thanks. I''ve been having strange problems setting up the networking in this Xen system since the beginning. I''ll try again from scratch -- though, at the moment, I''m not yet able to get a proper "routed" configuration to work at all, and I''m hoping someone over on the xen-users list will have some insight as to what I''m doing wrong. (I tried imitating the "Xen My Way-Routed" example in the Shorewall documentation, but for some reason it simply won''t work for me -- the domU stubbornly refuses to connect to the network and flatly will not start up.) -- Rich Wales === Palo Alto, CA, USA === richw@richw.org http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Rich Wales wrote:> Tom Eastep wrote: > > (I tried imitating the "Xen My > Way-Routed" example in the Shorewall documentation, but for some reason > it simply won''t work for me -- the domU stubbornly refuses to connect to > the network and flatly will not start up.) >As I point out in the XenMyWay-Routed doc, the ''out of the box'' routed domU configuration _will not_ connect to the network (it can connect to the dom0). You can correct that problem by doing this: echo 1 > /proc/sys/net/ipv4/conf/<external interface>/proxy_arp Note that Shorewall will do that for you once you have it properly configured. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Hi, Tom -- Replying to:> As I point out in the XenMyWay-Routed doc, the ''out of the box'' routed > domU configuration _will not_ connect to the network (it can connect to > the dom0). You can correct that problem by doing this: > > echo 1 > /proc/sys/net/ipv4/conf/<external interface>/proxy_arp > > Note that Shorewall will do that for you once you have it properly > configured.I know that, and I made sure that proxy ARP was enabled for the interface in question, as you''ve described. But as best I can tell, this is NOT the basis of the problems I''ve been running into. My problem is NOT an inability of working domU''s to connect to the network. Rather, my problem is that routing configuration problems have prevented domU''s from starting up AT ALL -- and when I finally did manage to kludge together a Xen networking setup where domU''s could be successfully created, a Shorewall running on the Xen box (in dom0) was unable to deal with the network traffic (as illustrated by the SNAT problem I described). The thing that''s really frustrating here is that even though I thought I was following your instructions for setting up a routed configuration, something was clearly going wrong, because it simply wouldn''t work (my domU''s claimed to be running into some unspecified networking setup error in vif-route and would not start). And when, after much trial and error, I did finally manage to put together something that appeared to work, I discovered that Shorewall simply wouldn''t behave properly with my setup (as you''ve seen). I''ve gone back to Square One and am trying, again, to get a vanilla routed Xen configuration (without any bridging whatsoever) to work. So far, I''m having no success; when I try to create a domU, vif-route logs a cryptic network setup error, and my "xm create" command exits without having accomplished a thing. Any suggestions are welcome. I''ve described my problem on xen-users, but so far at least, no one over there has come up with anything useful. -- Rich Wales === Palo Alto, CA, USA === richw@richw.org http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
OK, I finally managed to figure out what was going haywire in my Xen configuration. It turns out that if you have a routed Xen setup and are using a non-default network interface in dom0 (i.e., something other than eth0), you need to explicitly pass the interface name (via a netdev= parameter) to BOTH network-route AND vif-route. I hadn''t realized it was necessary to give the netdev= parameter to vif-route. After adding this parameter, I was able to set up Shorewall to do SNAT on outbound connections from my domU''s (absolutely essential if they were going to be able to connect out to the Internet). Hopefully this tidbit of knowledge can be mentioned in some FAQ''s and how-to''s, so other people won''t need to suffer the way I did (and perhaps just give up like I almost did). -- Rich Wales === Palo Alto, CA, USA === richw@richw.org http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Rich Wales wrote:> OK, I finally managed to figure out what was going haywire in my Xen > configuration. It turns out that if you have a routed Xen setup and > are using a non-default network interface in dom0 (i.e., something > other than eth0), you need to explicitly pass the interface name (via > a netdev= parameter) to BOTH network-route AND vif-route. I hadn''t > realized it was necessary to give the netdev= parameter to vif-route. > > After adding this parameter, I was able to set up Shorewall to do SNAT > on outbound connections from my domU''s (absolutely essential if they > were going to be able to connect out to the Internet). > > Hopefully this tidbit of knowledge can be mentioned in some FAQ''s and > how-to''s, so other people won''t need to suffer the way I did (and > perhaps just give up like I almost did). >This information is indeed important -- in Xen FAQs and HOWTOs. So I suggest that you post to those folks. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Tom Eastep wrote:> This information is indeed important -- in Xen FAQs and HOWTOs. > So I suggest that you post to those folks.Don''t worry, I did. (I thought it might be relevant here too, since you do have material on the Shorewall site about using Shorewall with Xen . . . but whatever . . . .) Now that I''ve got both SNAT and DNAT working in my configuration, I''m trying to tighten up the firewall so that clients outside my Xen box cannot contact the domU''s directly, but can access services ONLY by connecting to the dom0 and being DNAT''ed to the appropriate domU. In other words, my Xen box should look like a single host (the dom0) to all outsiders; the internal structure (with multiple domU''s) should be invisible and inaccessible from the outside. I started with my HTTP server (on "wonttell", 172.31.53.5). I tried to modify my Shorewall rules for HTTP so that hosts on my LAN can access the HTTP server only via DNAT from the dom0 ("whodunit", 172.29.0.53). However, the extra attempt at security doesn''t seem to be working -- connections from my LAN directly to the domU (wonttell) are NOT being blocked -- they are still getting through. The attached Shorewall dump should be capturing what happened when I did "telnet 172.31.53.5 http" (and successfully connected) from a host on my LAN (172.29.0.29). I''m confused that the dump doesn''t seem to show ANY PACKETS AT ALL being processed for port 80 on the domU (172.31.53.5). Is it possible that something is still broken with the networking in my Xen configuration, and that traffic between my LAN and my domU''s is completely bypassing Shorewall? I''d be more than willing to settle for some solution that would make my DMZ network (172.31.53.0/24, containing my domU''s) completely off limits from the outside for all services (even SSH -- meaning that I would need to SSH to the dom0 first and use it as a bastion host to get to the domU''s). I know Xen has a NAT networking configuration (an alternative to vanilla routing or bridging), though I haven''t tried it and suspect that since it seems to use iptables, it''s probably not compatible with Shorewall anyway. -- Rich Wales === Palo Alto, CA, USA === richw@richw.org http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Rich Wales wrote:> > The attached Shorewall dump should be capturing what happened when I > did "telnet 172.31.53.5 http" (and successfully connected) from a host > on my LAN (172.29.0.29). I''m confused that the dump doesn''t seem > to show ANY PACKETS AT ALL being processed for port 80 on the domU > (172.31.53.5). Is it possible that something is still broken with the > networking in my Xen configuration, and that traffic between my LAN and > my domU''s is completely bypassing Shorewall?No. You have not defined eth5:172.31.53.5 to be part of any zone. To compensate for this inadequate zone definition, you have configured an all->all policy of ACCEPT! That policy is allowing anything from anywhere to anywhere. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/