Shorewall users - Sep 2003 - Routing hell.

I''ll start by saying that I''m not convinced this is a
shorewall issue,
but...

I''m at a loss to explain this... hopefully someone can help.

I have a 3-interface setup (config below), and am using my firewall
(named ''fw'') as the default gateway for my network. The
internal
interface on my firewall is 172.16.127.254.

My main network is 172.16.0.0/17 (not a typo).

I have a lab network (172.17.0.0/16) hanging off another system named
''fiat'', which has two network addresses, 172.16.0.4 and
172.17.255.254.

Systems in the 172.17 network have 172.17.255.254 as their default
gateway. Systems in the 172.16 network have 172.16.127.254 (the
firewall) as their default gateway.

If I ping a system, say 172.17.0.3 from my desktop workstation at
172.16.1.1, I get an ICMP redirect from the firewall, and successfully
get through.

Looking at /proc/net/rt_cache shows correct dynamic routing entries,
indicating 172.16.0.4 as the gateway for 172.17.0.3, eg:

eth0 030011AC 040010AC .....

If instead of pinging first, I use ssh to try attaching, things hang. If
I look in /proc/net/rt_cache, I get multiple entries, some correct and
some indicating the firewall (172.16.127.254) as the gateway for
172.17.0.3, eg:

eth0 030011AC FE7F10AC .....

Now, the wierdness is that if I ping first, then shortly thereafter try
using ssh, everything connects and all is well. Even stranger, I can
successfully telnet to this system without getting the bogus entries in
rt_cache, it is only when I use secure shell that they appear. My ssh
client configuration is simple (X11 forwarding turned on, nothing else).
My sshd configuration on the lab system is equally simple. 

If I put in a static route for the 172.17 network on my system,
everything works fine.

And... if I use a Linux system other than my firewall as my default
router, everything also works fine, I get the proper redirects, rt_cache
contains what I''d expect (only entries for the correct 172.16.0.4
gateway). It seems that the redirects that get passed back by the
firewall, only while using ssh, are not valid for some reason.

My shorewall config:

interfaces:

-       eth0            detect
dmz     eth1            detect          dropunclean,tcpflags
net     eth2            detect          norfc1918,dropunclean,tcpflags

hosts:

svr             eth0:172.16.0.0/24
loc             eth0:172.16.0.0/17
lab             eth0:172.17.0.0/16

policy:

svr             all             CONTINUE
loc             dmz             REJECT
loc             net             ACCEPT
loc             lab             ACCEPT
lab             loc             ACCEPT
net             all             DROP            info
all             all             REJECT          info

zones:

svr     Server          Internal Servers
net     Net             Internet
loc     Local           Local Networks
lab     Lab             Lab zone
dmz     DMZ             Demilitarized Zone

rules:

ACCEPT  svr     fw      tcp     ssh
ACCEPT  svr     dmz     tcp     ssh
ACCEPT  fw      dmz     tcp     domain
ACCEPT  fw      dmz     udp     domain
DNAT    net     dmz:192.168.1.2 tcp     domain  # External named access
DNAT    net     dmz:192.168.1.2 udp     domain  # External named access
ACCEPT  dmz     net     tcp     domain
ACCEPT  dmz     net     udp     domain
ACCEPT  svr     dmz     tcp     domain
ACCEPT  svr     dmz     udp     domain
ACCEPT  loc     fw      icmp    8       # local can ping firewall
ACCEPT  dmz     fw      icmp    8       # dmz can ping firewall
ACCEPT  fw      loc     icmp    8       # firewall can ping local
ACCEPT  fw      dmz     icmp    8       # firewall can ping dmz
ACCEPT  fw      net     icmp    8       # firewall can ping net
ACCEPT  loc     dmz     icmp    8       # local can ping dmz
ACCEPT  dmz     net     icmp    8       # dmz can ping internet
ACCEPT  svr     dmz     tcp     smtp    # local can talk to dmz sendmail
ACCEPT  dmz     svr     tcp     smtp    # dmz can talk to local sendmail
ACCEPT  dmz     net     tcp     smtp    # dmz can talk to internet
sendmail
DNAT    net     dmz:192.168.1.2 tcp     smtp
ACCEPT  fw      svr     tcp     smtp
DNAT net:24.87.20.42    svr:172.16.0.1  tcp     ssh # ssh for CSD
DNAT net:24.80.108.46   svr:172.16.0.1  tcp     ssh # ssh for parker
DNAT net:24.82.143.12   svr:172.16.0.1  tcp     ssh # ssh for josefr
ACCEPT          fw      net     tcp     www
ACCEPT          fw      net     tcp     https
ACCEPT          loc     fw      tcp     squid
DNAT    net     dmz:192.168.1.2 tcp     www     # External www access to
trabant
ACCEPT  loc     dmz     tcp     www     # internal net access to trabant
ACCEPT  fw      dmz     tcp     www     # squid cache access to trabant
ACCEPT  dmz     net     tcp     ntp
ACCEPT  dmz     net     udp     ntp
ACCEPT  svr     dmz     tcp     ntp
ACCEPT  svr     dmz     udp     ntp
ACCEPT  fw      dmz     tcp     ntp
ACCEPT  fw      dmz     udp     ntp
ACCEPT  loc     fw      tcp     5666
ACCEPT  loc     dmz     tcp     5666


That''s it - everything else is default config.

Help!!!

Cheers,

RP


-- 
Ross Parker
OctigaBay Systems Corp.

phone: 604-484-2265
fax:   604-484-2221
cell:  604-817-3500

On Fri, 19 Sep 2003, Ross Parker wrote:
>
> If instead of pinging first, I use ssh to try attaching, things hang. If
> I look in /proc/net/rt_cache, I get multiple entries, some correct and
> some indicating the firewall (172.16.127.254) as the gateway for
> 172.17.0.3, eg:
>
Try setting NEWNOTSYN=Yes.

-Tom

Tom Eastep    \ Shorewall - iptables made easy
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net

Tom,
> > If instead of pinging first, I use ssh to try attaching, 
> things hang. 
> > If I look in /proc/net/rt_cache, I get multiple entries, 
> some correct 
> > and some indicating the firewall (172.16.127.254) as the 
> gateway for 
> > 172.17.0.3, eg:
> 
> Try setting NEWNOTSYN=Yes.
Wow - thanks. That did the trick beautifully (in a quick test from home, at
least).

I don''t understand though - why only with ssh? All other protocols I
tried
seemed to work fine. The firewall should only be sending redirects, not
actually passing the traffic, unless I''m misunderstanding something...

BTW, I should express my thanks for Shorewall (and obviously for your
support!) - great product.

Cheers,

Ross

On Fri, 19 Sep 2003, Ross Parker wrote:
> > 
> > Try setting NEWNOTSYN=Yes.
> 
> Wow - thanks. That did the trick beautifully (in a quick test from home, at
> least).
> 
> I don''t understand though - why only with ssh? All other protocols
I tried
> seemed to work fine. The firewall should only be sending redirects, not
> actually passing the traffic, unless I''m misunderstanding
something...
> 
NETNOTSYN applies to TCP (which is used by SSH). Because ICMP redirects
are being sent, the firewall only sees the traffic in one direction. Under
that condition. NETNOTSYN=No will stop any follow-on traffic in the same
direction as the original SYN and communication will fail.
> BTW, I should express my thanks for Shorewall (and obviously for your
> support!) - great product.
> 
You''re welcome.

-Tom
-- 
Tom Eastep    \ Shorewall - iptables made easy
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net

Hi, Tom,
> NETNOTSYN applies to TCP (which is used by SSH). Because ICMP 
> redirects are being sent, the firewall only sees the traffic 
> in one direction. Under that condition. NETNOTSYN=No will 
> stop any follow-on traffic in the same direction as the 
> original SYN and communication will fail.
I''m obviously misunderstanding the semantics of an ICMP redirect. I was
expecting that after the redirect was sent (and I did verify that with
ethereal), that all traffic would then remain off the firewall, the ssh
client (on the 172.16 net) would go straight through the system that routes
between the 172.16 and 172.17 networks (where the ssh server is). I
don''t
understand why the traffic would continue to flow through the firewall.

I''m also still confused as to why this only seemed to apply to ssh!
This all
came to a head today, incidentally, when the system that routes between my
172.16 and 172.17 networks was rebooted. For the life of me I cannot find
anything on it that is set up incorrectly (routes are fine, forwarding is
turned on, etc), but obviously the reboot has caused a change.

Regardless, I think I''ll rearrange my routing setup to not involve the
firewall in any case. It was in the loop solely to provide a redirect.

Thanks again for the help...

RP