Using shorewall-4.4.2-2 on two CentOS 5.4 x86_64 firewall boxes, using kernel 2.6.18-164.el5 and LVS-DR by way of keepalived-1.1.19. /etc/shorewall/interfaces: pub eth0 - tcpflags,nosmurfs,maclist,routeback net eth2 - tcpflags,routefilter,nosmurfs,logmartians,blacklist,arp_filter where "pub" is internal network 192.168.0.0/22, and eth2 is the outside world. I have several services defined in keepalived, including an SMTP service (192.168.0.20) that points to three internal servers. From ipvsadm: TCP 192.168.0.20:25 wrr -> 192.168.0.11:25 Route 50 0 0 -> 192.168.0.13:25 Route 100 0 0 -> 192.168.0.12:25 Route 100 0 0 where the three internal servers do not respond to ARP requests for 192.168.0.20.>From /sbin/ipvsadm:3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:14:22:27:99:6a brd ff:ff:ff:ff:ff:ff inet 192.168.0.5/22 brd 192.168.3.255 scope global eth0 ... inet 192.168.0.20/22 brd 192.168.3.255 scope global secondary eth0 Everything is working fine from the point of view of clients in the pub zone. I have two problems, and I have been unable to make either of these work: (1) I need to be able to make connections from the firewall itself to the 192.168.0.20:25 service. (2) I need to be able to DNAT incoming clients from the internet to the 192.168.0.20:25 service. Until I can get (1) to work, there''s little point in tackling (2). On the firewall, I do a "telnet 192.168.0.20 25" and get a connection refused, with this in the syslog: Oct 29 16:54:14 fw1 kernel: Shorewall:nat:OUTPUT:IN= OUT=lo SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 Oct 29 16:54:14 fw1 kernel: Shorewall:filter:OUTPUT:IN= OUT=lo SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 Oct 29 16:54:14 fw1 kernel: Shorewall:mangle:POSTROUTING:IN= OUT=lo SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 Oct 29 16:54:14 fw1 kernel: Shorewall:nat:POSTROUTING:IN= OUT=lo SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 Oct 29 16:54:14 fw1 kernel: Shorewall:mangle:PREROUTING:IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 Oct 29 16:54:14 fw1 kernel: Shorewall:mangle:INPUT:IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 Oct 29 16:54:14 fw1 kernel: Shorewall:filter:INPUT:IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 Rules: ACCEPT $FW pub tcp smtp ACCEPT pub $FW tcp smtp Someone hit me with a clue stick and tell me what rules I am missing. Steve ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Steve Thompson wrote:> Using shorewall-4.4.2-2 on two CentOS 5.4 x86_64 firewall boxes, using > kernel 2.6.18-164.el5 and LVS-DR by way of keepalived-1.1.19.There are a number of issues with running a Netfilter-based firewall on an LVS-DR director. Chief among them is that connection-tracking doesn''t work properly. We suggest placing a separate firewall between the LVS director and servers and the upstream router. Router-->FW--->director---->servers ^ | |_____ replies _____|> > /etc/shorewall/interfaces: > pub eth0 - tcpflags,nosmurfs,maclist,routeback > net eth2 - tcpflags,routefilter,nosmurfs,logmartians,blacklist,arp_filter > > where "pub" is internal network 192.168.0.0/22, and eth2 is the outside > world. > > I have several services defined in keepalived, including an SMTP service > (192.168.0.20) that points to three internal servers. From ipvsadm: > > TCP 192.168.0.20:25 wrr > -> 192.168.0.11:25 Route 50 0 0 > -> 192.168.0.13:25 Route 100 0 0 > -> 192.168.0.12:25 Route 100 0 0 > > where the three internal servers do not respond to ARP requests for > 192.168.0.20. > >>From /sbin/ipvsadm: > > 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 > link/ether 00:14:22:27:99:6a brd ff:ff:ff:ff:ff:ff > inet 192.168.0.5/22 brd 192.168.3.255 scope global eth0 > ... > inet 192.168.0.20/22 brd 192.168.3.255 scope global secondary eth0 > > Everything is working fine from the point of view of clients in the pub > zone. I have two problems, and I have been unable to make either of these > work: > > (1) I need to be able to make connections from the firewall itself to the > 192.168.0.20:25 service. > > (2) I need to be able to DNAT incoming clients from the internet to the > 192.168.0.20:25 service. Until I can get (1) to work, there''s little point > in tackling (2). > > On the firewall, I do a "telnet 192.168.0.20 25" and get a connection > refused, with this in the syslog: > > Oct 29 16:54:14 fw1 kernel: Shorewall:nat:OUTPUT:IN= OUT=lo SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 > Oct 29 16:54:14 fw1 kernel: Shorewall:filter:OUTPUT:IN= OUT=lo SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 > Oct 29 16:54:14 fw1 kernel: Shorewall:mangle:POSTROUTING:IN= OUT=lo SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 > Oct 29 16:54:14 fw1 kernel: Shorewall:nat:POSTROUTING:IN= OUT=lo SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 > Oct 29 16:54:14 fw1 kernel: Shorewall:mangle:PREROUTING:IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 > Oct 29 16:54:14 fw1 kernel: Shorewall:mangle:INPUT:IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 > Oct 29 16:54:14 fw1 kernel: Shorewall:filter:INPUT:IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=192.168.0.20 DST=192.168.0.20 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41216 DF PROTO=TCP SPT=53768 DPT=25 WINDOW=32792 RES=0x00 SYN URGP=0 > > Rules: > ACCEPT $FW pub tcp smtp > ACCEPT pub $FW tcp smtp > > Someone hit me with a clue stick and tell me what rules I am missing.This is behaving as if 192.168.0.20 is not being redirected and the connection is entirely local to the firewall itself (note that the connection is being routed through the loopback device and is a fw->fw connection, not a fw->pub connection). I suspect that there is no SMTP server running on the firewall which is why you are getting connection refused. In other words, the problem has nothing to do with Shorewall. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
On Thu, 29 Oct 2009, Tom Eastep wrote:> This is behaving as if 192.168.0.20 is not being redirected and the > connection is entirely local to the firewall itself (note that the > connection is being routed through the loopback device and is a fw->fw > connection, not a fw->pub connection). I suspect that there is no SMTP > server running on the firewall which is why you are getting connection > refused. In other words, the problem has nothing to do with Shorewall.Tom, Thanks for your insight and the clue stick. You are of course correct; there _was_ a SMTP server running on the firewall, but it was listening on the loopback address only. Duh. I know better than this. Really. Steve ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
On Thu, 29 Oct 2009, Tom Eastep wrote:> This is behaving as if 192.168.0.20 is not being redirected and the > connection is entirely local to the firewall itself (note that the > connection is being routed through the loopback device and is a fw->fw > connection, not a fw->pub connection).Any way to DNAT a connection from the internet to a keepalived VIP, where the VIP is held on one of the firewall''s internal interfaces? -s ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Steve Thompson wrote:> On Thu, 29 Oct 2009, Tom Eastep wrote: > >> This is behaving as if 192.168.0.20 is not being redirected and the >> connection is entirely local to the firewall itself (note that the >> connection is being routed through the loopback device and is a fw->fw >> connection, not a fw->pub connection). > > Any way to DNAT a connection from the internet to a keepalived VIP, > where the VIP is held on one of the firewall''s internal interfaces?Not that I''m aware of. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference