Glenn St. John
2004-Oct-21 14:58 UTC
After shorewall restart NAT SMTP connection slow; reboot and it works fine
I recently implemented v2.0.9 using ''shorewall setup guide'' 2004-07-31. Starting with block everything not known to be in use and opening ports as complaints come in. This has led to a few rule changes. After a rule change I use shorewall restart to reload the rules. Seems to work OK... except for an outbound NAT SMTP connection from a mail server on .122 to postini.com. The SMTP server can connect and transfer messages but does so very slowly. If I reboot the firewall (after the changed rules are saved) SMTP flows normally. (Stopping and restarting, or even rebooting the SMTP server doesn''t help.) After perusing the Documentation Index I was wondering if instead of ''shorewall restart'' the command to use would be ''shorewall refresh'', or maybe ''shorewall stop; shorewall clear; shorewall start''. It seems as if there is maybe a state or translation table somewhere that isn''t being cleared (or something like that) but that is cleared by a reboot. Any help sincerely appreciated. I would prefer not to reboot the firewall every time there is a rule change, although once we get through another week there shouldn''t be the same volume of rule changes. Using shorewall 2.0.9 on latest Bering (LEAF), downloaded updated shorewall.lrp from link on shorewall site ip addr show: 1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo 2: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 3: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:50:8b:e0:7d:15 brd ff:ff:ff:ff:ff:ff inet 12.4.123.114/28 brd 12.4.123.127 scope global eth0 inet 12.4.123.118/28 brd 12.4.123.127 scope global secondary eth0:0 inet 12.4.123.122/28 brd 12.4.123.127 scope global secondary eth0:0 4: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:50:8b:e0:7d:14 brd ff:ff:ff:ff:ff:ff inet 172.16.2.254/16 brd 172.16.255.255 scope global eth1 ip route: 12.4.123.112/28 dev eth0 proto kernel scope link src 12.4.123.114 172.16.0.0/16 dev eth1 proto kernel scope link src 172.16.2.254 172.16.0.0/13 via 172.16.1.1 dev eth1 default via 12.4.123.113 dev eth0 Tom - thanks for Shorewall. Glenn St. John THIS MESSAGE contains information that is CONFIDENTIAL and legally exempt from disclosure and that is intended only for the addressee of this message. IF YOU are not the addressee or a person responsible for delivering this message to the addressee, then do not copy or deliver this message to anyone and please delete this message and contact us at +1.703.836.6620. Thank you.
Tom Eastep
2004-Oct-21 15:43 UTC
Re: After shorewall restart NAT SMTP connection slow; reboot and it works fine
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Glenn St. John wrote:> I recently implemented v2.0.9 using ''shorewall setup guide'' 2004-07-31. > > Starting with block everything not known to be in use and opening ports > as complaints come in. This has led to a few rule changes. After a > rule change I use shorewall restart to reload the rules. Seems to work > OK... except for an outbound NAT SMTP connection from a mail server on > .122 to postini.com. The SMTP server can connect and transfer messages > but does so very slowly. If I reboot the firewall (after the changed > rules are saved) SMTP flows normally. (Stopping and restarting, or even > rebooting the SMTP server doesn''t help.) After perusing the > Documentation Index I was wondering if instead of ''shorewall restart'' > the command to use would be ''shorewall refresh'', or maybe ''shorewall > stop; shorewall clear; shorewall start''. It seems as if there is maybe > a state or translation table somewhere that isn''t being cleared (or > something like that) but that is cleared by a reboot. > > Any help sincerely appreciated. I would prefer not to reboot the > firewall every time there is a rule change, although once we get through > another week there shouldn''t be the same volume of rule changes. >The "shorewall restart" command completely reloads the Netfilter configuration and all of us who use Shorewall will routinely alter our ruleset and use "shorewall restart" to activate the change. To my knowledge, no one has seen ever reported the behavior that you are seeing. If you can reproduce this behavior again, I suggest that you: a) Note the conntrack table entry/entries for the slow connection(s) (from "shorewall show connections"). b) look at the slow connection with a packet sniffer such as Ethereal or tcpdump. That may give you a hint. c) After the reboot, and while mail is flowing normally look at the conntrack table entry/entries again. I mention the connection tracking table because that table is *not* cleared by a "shorewall restart" but an incorrect conntrack table entry usually causes things to not work at all and would not produce the symptoms that you report. And before you ask "why doesn''t shorewall clear the table during restart", there are 100s of posts covering this topic on the netfilter user''s list; basically, there is no reliable way to do it other than to reboot. - -Tom - -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBd9kVO/MAbZfjDLIRArgUAJ461QwcRNnVwypD88zAP0kxPyzNTACgkCtV 3D6ZF+AgUoiUE2szmLjFOZ0=I5MN -----END PGP SIGNATURE-----
Tom Eastep
2004-Oct-21 16:55 UTC
Re: After shorewall restart NAT SMTP connection slow; reboot and it works fine
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tom Eastep wrote:> > > The "shorewall restart" command completely reloads the Netfilter > configuration and all of us who use Shorewall will routinely alter our > ruleset and use "shorewall restart" to activate the change. To my > knowledge, no one has seen ever reported the behavior that you are seeing. > > If you can reproduce this behavior again, I suggest that you: > > a) Note the conntrack table entry/entries for the slow connection(s) > (from "shorewall show connections"). > b) look at the slow connection with a packet sniffer such as Ethereal or > tcpdump. That may give you a hint. > c) After the reboot, and while mail is flowing normally look at the > conntrack table entry/entries again. > > I mention the connection tracking table because that table is *not* > cleared by a "shorewall restart" but an incorrect conntrack table entry > usually causes things to not work at all and would not produce the > symptoms that you report. > > And before you ask "why doesn''t shorewall clear the table during > restart", there are 100s of posts covering this topic on the netfilter > user''s list; basically, there is no reliable way to do it other than to > reboot.Another possibility is that "shorewall restart" is altering your configuration in some way that you don''t expect. Do you make use of ADD_SNAT_ALIASES or ADD_IP_ALIASES in shorewall.conf? - -Tom _______________________________________________ Shorewall-users mailing list Post: Shorewall-users@lists.shorewall.net Subscribe/Unsubscribe: https://lists.shorewall.net/mailman/listinfo/shorewall-users Support: http://www.shorewall.net/support.htm FAQ: http://www.shorewall.net/FAQ.htm - -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBd+oOO/MAbZfjDLIRAtziAKCig0Px4RbseR4M4attc/a2eCmczACgwwAO q2k7YxI6MExcFJzn2dx6j58=qNdV -----END PGP SIGNATURE-----
Tom Eastep
2004-Oct-21 17:25 UTC
Re: After shorewall restart NAT SMTP connection slow; reboot and it works fine
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tom Eastep wrote:> > Another possibility is that "shorewall restart" is altering your > configuration in some way that you don''t expect. Do you make use of > ADD_SNAT_ALIASES or ADD_IP_ALIASES in shorewall.conf? >You can see what "shorewall [re]start" does in addition to setting up Netfilter by looking at /var/lib/shorewall/restore-base. That script is created during "shorewall [re]start" and is combined with the output of "iptables-save" by the "shorewall save" command. - -Tom - -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBd/D3O/MAbZfjDLIRAipTAJ9VUKuegjk7a2woxx/u2hye7zOBQQCeNc3+ EccbfyfdlBzIvCk0Rpga8+M=ME73 -----END PGP SIGNATURE-----
Glenn St. John
2004-Oct-21 19:16 UTC
After shorewall restart NAT SMTP connection slow; reboot and it works fine
> If you can reproduce this behavior again, I suggest that you: > > a) Note the conntrack table entry/entries for the slow connection(s) > (from "shorewall show connections").It seems reproducible (three times it has happened) so I will check the conntrack table and use ethereal to look at some packets.>Another possibility is that "shorewall restart" is altering your >configuration in some way that you don''t expect. Do you make use of >ADD_SNAT_ALIASES or ADD_IP_ALIASES in shorewall.conf?ADD_IP_ALIASES=Yes in shorewall.conf, (ADD_SNAT_ALIASES=No) In some way restarting is breaking this or the alias doesn''t work correctly after the restart? (I don''t follow where you are headed.) I guess the hint you gave me was to look at /var/lib/shorewall/restore-base, I see under ''restoring one-to-one NAT..'' it deletes the addresses? and then under ''restoring ip addresses...'' it adds them back ? In all likelihood I''m mis-reading the script so any help there appreciated.> And before you ask "why doesn''t shorewall clear the table during > restart", there are 100s of posts covering this topic on the netfilterSimple things should be simple and complex things possible (I read that somewhere) but I don''t expect shorewall to do something impossible. If it turns out in my environment, on leaf, with NAT, etc - that the answer is to reboot to clear the conntrack table to implement a rule change then so be it. It takes about 2 minutes to reboot which is longer than reloading but still not too bad. Glenn St. John THIS MESSAGE contains information that is CONFIDENTIAL and legally exempt from disclosure and that is intended only for the addressee of this message. IF YOU are not the addressee or a person responsible for delivering this message to the addressee, then do not copy or deliver this message to anyone and please delete this message and contact us at +1.703.836.6620. Thank you.
Tom Eastep
2004-Oct-21 20:01 UTC
Re: After shorewall restart NAT SMTP connection slow; reboot and it works fine
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Glenn St. John wrote:> > > ADD_IP_ALIASES=Yes in shorewall.conf, (ADD_SNAT_ALIASES=No) > In some way restarting is breaking this or the alias doesn''t work > correctly after the restart? (I don''t follow where you are headed.) I > guess the hint you gave me was to look at > /var/lib/shorewall/restore-base, I see under ''restoring one-to-one > NAT..'' it deletes the addresses? and then under ''restoring ip > addresses...'' it adds them back ? In all likelihood I''m mis-reading the > script so any help there appreciated.That''s what is happening -- so the real key is "which address(es) are being deleted?". If these are addresses with routes associated with them then when Shorewall adds back the address, it has no way of adding back the route. So in particular, if you try to use one-to-one NAT on the primary IP address of an external interface, these sorts of problems occur.> > >>And before you ask "why doesn''t shorewall clear the table during >>restart", there are 100s of posts covering this topic on the netfilter > > > Simple things should be simple and complex things possible (I read that > somewhere) but I don''t expect shorewall to do something impossible. If > it turns out in my environment, on leaf, with NAT, etc - that the answer > is to reboot to clear the conntrack table to implement a rule change > then so be it. It takes about 2 minutes to reboot which is longer than > reloading but still not too bad.As I said before, no one else that I''m aware of is rebooting to install a new ruleset -- I certainly wouldn''t put up with doing that. - -Tom - -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBeBWeO/MAbZfjDLIRArizAKCrwJwNbRUSfDPOpObiq5iOQSwsygCgjmaf sETORKeeWNFtxOxRktZuXi8=0H8g -----END PGP SIGNATURE-----
Tom Eastep
2004-Oct-21 20:03 UTC
Re: After shorewall restart NAT SMTP connection slow; reboot and it works fine
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tom Eastep wrote:> Glenn St. John wrote: > > >>> >>>ADD_IP_ALIASES=Yes in shorewall.conf, (ADD_SNAT_ALIASES=No) >>>In some way restarting is breaking this or the alias doesn''t work >>>correctly after the restart? (I don''t follow where you are headed.) I >>>guess the hint you gave me was to look at >>>/var/lib/shorewall/restore-base, I see under ''restoring one-to-one >>>NAT..'' it deletes the addresses? and then under ''restoring ip >>>addresses...'' it adds them back ? In all likelihood I''m mis-reading the >>>script so any help there appreciated. > > > That''s what is happening -- so the real key is "which address(es) are > being deleted?". If these are addresses with routes associated with them > then when Shorewall adds back the address, it has no way of adding back > the route. So in particular, if you try to use one-to-one NAT on the > primary IP address of an external interface, these sorts of problemsoccur.> > >>> >>>>And before you ask "why doesn''t shorewall clear the table during >>>>restart", there are 100s of posts covering this topic on the netfilter >>> >>> >>>Simple things should be simple and complex things possible (I read that >>>somewhere) but I don''t expect shorewall to do something impossible. If >>>it turns out in my environment, on leaf, with NAT, etc - that the answer >>>is to reboot to clear the conntrack table to implement a rule change >>>then so be it. It takes about 2 minutes to reboot which is longer than >>>reloading but still not too bad.So I would compare: a) The output of "ip addr ls" before and after you restart Shorewall. b) The output of "ip route ls" before and after you restart Shorewall. - -Tom - -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBeBYCO/MAbZfjDLIRAl9NAKC6Jg4V5jBMSyH3GxUmrgy5qZuC7gCdE/TE L5rpOGwaDZ7hdGbf0DSAtfA=/2w9 -----END PGP SIGNATURE-----