Brian J. Murrell
2013-Oct-21 14:57 UTC
correct mechanism to blacklist/drop only incoming connections
So, I have started to use fail2ban with shorewall. It looks like the correct mechanism to use is shorewall drop along with BLACKLISTNEWONLY=No. The problem is that that ends up blocking the inbound traffic from locally generated outbound sessions. My goal here is to add a drop/blacklist rule, dynamically, when fail2ban detects a host is doing bad things. A portscan is a good example to use for "bad things". But I also want to block/drop any established "inbound" connections that are open, for example to shut down a brute-force attacking of SSH that might have started before the portscan started. The collateral damage here though is for example, IRC servers that I want to connect to that want to do a portscan of me to make sure I''m not an open relay. They end up blacklisted in the dynamic chain with no regard to whether the packets it''s dropping are from outbound connections or inbound. Maybe the problem is that I''m wanting to have my cake and eat it too since there is no concept of inbound/outbound with ESTABLISHED sessions. Thoughts? b. ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
Tom Eastep
2013-Oct-21 15:50 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 10/21/2013 7:57 AM, Brian J. Murrell wrote:> So, I have started to use fail2ban with shorewall. It looks like the > correct mechanism to use is shorewall drop along with BLACKLISTNEWONLY=No. > > The problem is that that ends up blocking the inbound traffic from > locally generated outbound sessions. > > My goal here is to add a drop/blacklist rule, dynamically, when fail2ban > detects a host is doing bad things. A portscan is a good example to use > for "bad things". But I also want to block/drop any established > "inbound" connections that are open, for example to shut down a > brute-force attacking of SSH that might have started before the portscan > started. > > The collateral damage here though is for example, IRC servers that I > want to connect to that want to do a portscan of me to make sure I''m not > an open relay. They end up blacklisted in the dynamic chain with no > regard to whether the packets it''s dropping are from outbound > connections or inbound. > > Maybe the problem is that I''m wanting to have my cake and eat it too > since there is no concept of inbound/outbound with ESTABLISHED sessions. > > Thoughts?Try this: /etc/shorewall/actions ban fail2ban /etc/shorewall/action.fail2ban ban - - /etc/shorewall/action.ban is empty /etc/shorewall/rules ?section ESTABLISHED fail2ban net all ?section NEW fail2ban net all Now have fail2ban add DROP or REJECT rules to the ban chain -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
Brian J. Murrell
2013-Oct-21 18:02 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 13-10-21 11:50 AM, Tom Eastep wrote: Wow. Super solution Tom! Not surprised, but just wanted to say it anyway. :-)> Now have fail2ban add DROP or REJECT rules to the ban chainJust so I''m perfectly clear, "add ... rules" as in using iptables directly, yes? Just making sure there was not some shorewall-built-in facility for adding new rules dynamically that I was not aware of. As always, cheers, b. ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
Tom Eastep
2013-Oct-21 18:26 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 10/21/2013 11:02 AM, Brian J. Murrell wrote:> On 13-10-21 11:50 AM, Tom Eastep wrote: > > Wow. Super solution Tom! Not surprised, but just wanted to say it > anyway. :-) > >> Now have fail2ban add DROP or REJECT rules to the ban chain > > Just so I''m perfectly clear, "add ... rules" as in using iptables > directly, yes? Just making sure there was not some shorewall-built-in > facility for adding new rules dynamically that I was not aware of.That''s correct -- just run iptables directly. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Brian J. Murrell
2013-Oct-21 18:59 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 13-10-21 02:26 PM, Tom Eastep wrote:> > That''s correct -- just run iptables directly.One thing I did think of that the dynamic chain has that this solution doesn''t and that''s persistence of entries across a restart/reload. I suppose I could log banned entries in addition to installing the iptables rule and then create a start action to reload those from the log. Is there any more elegant way to do it? Cheers, b. ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Tom Eastep
2013-Oct-21 19:56 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 10/21/2013 11:59 AM, Brian J. Murrell wrote:> On 13-10-21 02:26 PM, Tom Eastep wrote: >> >> That''s correct -- just run iptables directly. > > One thing I did think of that the dynamic chain has that this solution > doesn''t and that''s persistence of entries across a restart/reload. > > I suppose I could log banned entries in addition to installing the > iptables rule and then create a start action to reload those from the log. > > Is there any more elegant way to do it? >You could keep a small file and reload from that in /etc/shorewall/start. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Brian J. Murrell
2013-Oct-21 20:47 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 13-10-21 03:56 PM, Tom Eastep wrote:> > You could keep a small file and reload from that in /etc/shorewall/start.Yeah, that''s basically what I was describing above. Still it might be an interesting feature to be able to save and restore the contents of an action (i.e. a chain) through start/stop/restart, etc. Thanks! b. ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Thomas D.
2013-Oct-21 21:53 UTC
Re: correct mechanism to blacklist/drop only incoming connections
Hi, Brian J. Murrell wrote:>> You could keep a small file and reload from that in /etc/shorewall/start. > > Yeah, that''s basically what I was describing above. Still it might be > an interesting feature to be able to save and restore the contents of an > action (i.e. a chain) through start/stop/restart, etc.Why don''t you use fail2ban with ipset? ipsets offer better performance over multiple rules. Most distributions are shipping scripts with their ipset packages, which will save/restore created ipsets on shutdown/start. If not, shorewall offers ipset saving/restoring support. -Thomas ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Tom Eastep
2013-Oct-21 22:03 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 10/21/2013 2:53 PM, Thomas D. wrote:> Hi, > > Brian J. Murrell wrote: >>> You could keep a small file and reload from that in /etc/shorewall/start. >> >> Yeah, that''s basically what I was describing above. Still it might be >> an interesting feature to be able to save and restore the contents of an >> action (i.e. a chain) through start/stop/restart, etc. > > Why don''t you use fail2ban with ipset? > > ipsets offer better performance over multiple rules. > > Most distributions are shipping scripts with their ipset packages, which > will save/restore created ipsets on shutdown/start. > If not, shorewall offers ipset saving/restoring support.Much better suggestion than mine! -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Brian J. Murrell
2013-Oct-22 10:53 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 13-10-21 11:50 AM, Tom Eastep wrote:> > Try this: > > /etc/shorewall/actions > > ban > fail2ban > > /etc/shorewall/action.fail2ban > ban - - > > /etc/shorewall/action.ban is empty > > /etc/shorewall/rules > > ?section ESTABLISHED > > fail2ban net all > > ?section NEW > > fail2ban net allI have noted later in the thread to use ipsets and this is a good idea, indeed. but before we add another complication I just wanted to report that this is not quite working. I think I know why. Here''s the net2loc chain for example: Chain net2loc (1 references) pkts bytes target prot opt in out source destination 528K 175M fail2ban all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate ESTABLISHED 528K 175M ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate ESTABLISHED 7984 752K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED 7340 967K fail2ban all -- * * 0.0.0.0/0 0.0.0.0/0 The problem as you can see is that while this does route established sessions through the [fail2]ban chain[s] as desired it''s also not allowing a new outbound session to get ESTABLISHed to a host that''s in the ban chains, I believe because the SYN-ACK from the remote (to which we are trying to ESTABLISH a connection to) that is needed to get the connection into the ESTABLISHed state is being sent to the ban chains by the 4th rule. Thoughts? Cheers, b. ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Tom Eastep
2013-Oct-22 14:15 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 10/22/2013 3:53 AM, Brian J. Murrell wrote:> On 13-10-21 11:50 AM, Tom Eastep wrote: >> >> Try this: >> >> /etc/shorewall/actions >> >> ban >> fail2ban >> >> /etc/shorewall/action.fail2ban >> ban - - >> >> /etc/shorewall/action.ban is empty >> >> /etc/shorewall/rules >> >> ?section ESTABLISHED >> >> fail2ban net all >> >> ?section NEW >> >> fail2ban net all > > I have noted later in the thread to use ipsets and this is a good idea, > indeed. but before we add another complication I just wanted to report > that this is not quite working. I think I know why. Here''s the net2loc > chain for example: > > Chain net2loc (1 references) > pkts bytes target prot opt in out source > destination > 528K 175M fail2ban all -- * * 0.0.0.0/0 > 0.0.0.0/0 ctstate ESTABLISHED > 528K 175M ACCEPT all -- * * 0.0.0.0/0 > 0.0.0.0/0 ctstate ESTABLISHED > 7984 752K ACCEPT all -- * * 0.0.0.0/0 > 0.0.0.0/0 ctstate RELATED > 7340 967K fail2ban all -- * * 0.0.0.0/0 > 0.0.0.0/0 > > The problem as you can see is that while this does route established > sessions through the [fail2]ban chain[s] as desired it''s also not > allowing a new outbound session to get ESTABLISHed to a host that''s in > the ban chains, I believe because the SYN-ACK from the remote (to which > we are trying to ESTABLISH a connection to) that is needed to get the > connection into the ESTABLISHed state is being sent to the ban chains by > the 4th rule.The plot thickens. The ''conntrack'' match has the ability to distinguish the original direction from the reply direction using ''--cttdir [ORIGINAL|REPLY]''. So I guess that you could could place this as the first rule of the fail2ban action: INLINE(CONTINUE) - - ; -m conntrack --ctdir REPLY -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Brian J. Murrell
2013-Oct-22 15:29 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 13-10-22 10:15 AM, Tom Eastep wrote:> The plot thickens.Sorry. I thought that requirement was described in my original message. Clearly I was not detailed enough in my description. Sorry about that.> The ''conntrack'' match has the ability to distinguish > the original direction from the reply direction using ''--cttdir > [ORIGINAL|REPLY]''. So I guess that you could could place this as the > first rule of the fail2ban action: > > INLINE(CONTINUE) - - ; -m conntrack --ctdir REPLYCompiling /etc/shorewall/gw-new-AA/action.fail2ban for chain fail2ban... ERROR: Invalid column/value pair (-m) : /etc/shorewall/gw-new-AA/action.fail2ban (line 1) Maybe my shorewall is not new enough. 4.4.26.1 on Ubuntu LTS. 4.4.27.3 on the remote (i.e. shorewall-lite) machine. My whole action.fail2ban just to double-check: INLINE(CONTINUE) - - ; -m conntrack --ctdir REPLY ban - - ban_repeater - - I have two different chains I want fail2ban to add rules to depending on if they are repeat offenders or not. Cheers, b. ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Tom Eastep
2013-Oct-22 15:49 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 10/22/2013 8:29 AM, Brian J. Murrell wrote:> On 13-10-22 10:15 AM, Tom Eastep wrote: >> The plot thickens. > > Sorry. I thought that requirement was described in my original message. > Clearly I was not detailed enough in my description. Sorry about that. >Not at all -- It was my failure to think this through completely.>> The ''conntrack'' match has the ability to distinguish >> the original direction from the reply direction using ''--cttdir >> [ORIGINAL|REPLY]''. So I guess that you could could place this as the >> first rule of the fail2ban action: >> >> INLINE(CONTINUE) - - ; -m conntrack --ctdir REPLY > > Compiling /etc/shorewall/gw-new-AA/action.fail2ban for chain fail2ban... > ERROR: Invalid column/value pair (-m) : > /etc/shorewall/gw-new-AA/action.fail2ban (line 1) > > Maybe my shorewall is not new enough. 4.4.26.1 on Ubuntu LTS. 4.4.27.3 > on the remote (i.e. shorewall-lite) machine.Much too old. 4.4.26 was released almost two years ago. INLINE was introduced this spring in 4.5.16, -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Brian J. Murrell
2013-Oct-22 16:26 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 13-10-22 11:49 AM, Tom Eastep wrote:> > Much too old. 4.4.26 was released almost two years ago.Yeah, that''s the unfortunate side of wanting to get off the upgrade-every-six-months treadmill. Damned if you do, damned if you don''t.> INLINE was > introduced this spring in 4.5.16,Does this solution require INLINE? Any alternative solutions (other than upgrading shorewall)? FWIW, simply removing the INLINE() still results in ERROR: Invalid column/value pair (-m) : /etc/shorewall/gw-new-AA/action.fail2ban (line 1) Cheers, b. ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Tom Eastep
2013-Oct-22 19:52 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 10/22/2013 9:26 AM, Brian J. Murrell wrote:> On 13-10-22 11:49 AM, Tom Eastep wrote: >> >> Much too old. 4.4.26 was released almost two years ago. > > Yeah, that''s the unfortunate side of wanting to get off the > upgrade-every-six-months treadmill. Damned if you do, damned if you don''t. > >> INLINE was >> introduced this spring in 4.5.16, > > Does this solution require INLINE? Any alternative solutions (other > than upgrading shorewall)? > > FWIW, simply removing the INLINE() still results in > > ERROR: Invalid column/value pair (-m) : > /etc/shorewall/gw-new-AA/action.fail2ban (line 1)Only other way is to inject the rule manually in your /etc/shorewall/start script. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Brian J. Murrell
2013-Oct-23 04:23 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 13-10-21 05:53 PM, Thomas D. wrote:> Hi,Hi Thomas,> Why don''t you use fail2ban with ipset?Good idea. I looked into this. It seems this simplifies the problem quite a bit, if I''ve got it right.> ipsets offer better performance over multiple rules.Indeed. So, just to make sure I have this right, disregard all of the previously discussed solution WRT to actions, etc. and simply add to the shorewall blacklist file: +fail2ban +fail2ban_perm and then create the two ipsets with: ipset -N fail2ban_perm iphash ipset -N fail2ban iphash and then have fail2ban simply add and remove entries from the ipset, correct? I can achieve my goal of allowing connections *to* blacklisted sites to be permitted by adding the following hack to started: $IPTABLES -I blacklst -m conntrack --ctdir REPLY -j RETURN and I''m seem to be good to go. Do I have that all correct? As an aside I wonder if it''s a worthwhile feature to have a setting to allow connections *to* blacklisted sites and to add the above to the blacklst chain if that setting is enabled. What do you think Tom? Cheers, b. ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Thomas D.
2013-Oct-23 13:06 UTC
Re: correct mechanism to blacklist/drop only incoming connections
Hi, Brian J. Murrell wrote:> So, just to make sure I have this right, disregard all of the previously > discussed solution WRT to actions, etc. and simply add to the shorewall > blacklist file: > > +fail2ban > +fail2ban_perm > > and then create the two ipsets with: > > ipset -N fail2ban_perm iphash > ipset -N fail2ban iphash > > and then have fail2ban simply add and remove entries from the ipset, > correct? > > I can achieve my goal of allowing connections *to* blacklisted sites to > be permitted by adding the following hack to started: > > $IPTABLES -I blacklst -m conntrack --ctdir REPLY -j RETURN > > and I''m seem to be good to go. > > Do I have that all correct?Well, I don''t like to change the way any existing features work: - You have to remember, that you have changed something. - If you ever want to use a ''traditional'' blacklist, you cannot. - Any changes upstream will make may break your firewall. So I wouldn''t try to change the blacklist feature, instead I would implement my own custom blacklist (using the rules file or whatever is needed but without manipulating any existing rules). But this is just a design question. If you are fine with your changes and they work the way you want it, go with it. Two annotations:> My goal here is to add a drop/blacklist rule, dynamically, when fail2ban > detects a host is doing bad things. A portscan is a good example to use > for "bad things". But I also want to block/drop any established > "inbound" connections that are open, for example to shut down a > brute-force attacking of SSH that might have started before the portscan > started.Have a look at the deprecated BLACKLISTNEWONLY or the new BLACKLIST option: http://www.shorewall.net/manpages/shorewall.conf.html> The collateral damage here though is for example, IRC servers that I > want to connect to that want to do a portscan of me to make sure I''m not > an open relay. They end up blacklisted in the dynamic chain with no > regard to whether the packets it''s dropping are from outbound > connections or inbound.That sounds like you are trying to fix the wrong problem: If a legitimate source will get blacklisted while doing an authorized port scan, your portscan rules are way too sensitive.> As an aside I wonder if it''s a worthwhile feature to have a setting to > allow connections *to* blacklisted sites and to add the above to the > blacklst chain if that setting is enabled.I am still wondering why somebody wants to block every connection attempt from a source but still want to be able to contact that source at all. Can you share more use cases with us? -Thomas ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Brian J. Murrell
2013-Oct-23 13:44 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 13-10-23 09:06 AM, Thomas D. wrote:> Hi,Hi,> Well, I don''t like to change the way any existing features work: > > - You have to remember, that you have changed something. > > - If you ever want to use a ''traditional'' blacklist, you cannot. > > - Any changes upstream will make may break your firewall.All fair points.> Two annotations: > >> My goal here is to add a drop/blacklist rule, dynamically, when fail2ban >> detects a host is doing bad things. A portscan is a good example to use >> for "bad things". But I also want to block/drop any established >> "inbound" connections that are open, for example to shut down a >> brute-force attacking of SSH that might have started before the portscan >> started. > > Have a look at the deprecated BLACKLISTNEWONLYI''m already using this with =No.> or the new BLACKLIST > option: http://www.shorewall.net/manpages/shorewall.conf.htmlProbably not available in the 4.4.26.1 that I''m using on Ubuntu LTS. Yeah: Added in Shorewall 4.5.13 to replace the BLACKLISTNEWONLY option below.>> The collateral damage here though is for example, IRC servers that I >> want to connect to that want to do a portscan of me to make sure I''m not >> an open relay. They end up blacklisted in the dynamic chain with no >> regard to whether the packets it''s dropping are from outbound >> connections or inbound. > > That sounds like you are trying to fix the wrong problem: > > If a legitimate source will get blacklisted while doing an authorized > port scan, your portscan rules are way too sensitive.Yes. Ideally, handling the portscanning that one (however reluctantly) agrees to when one connects, say, to an IRC server as "authorized portscanning" in opposition to unauthorized port scanning is the way to go, however I don''t want to have to play the cat & mouse game of deciding who is authorized to do port scans because of what my users are doing and who is just rattling door knobs randomly. I have better things to do with my time.> I am still wondering why somebody wants to block every connection > attempt from a source but still want to be able to contact that source > at all. Can you share more use cases with us?I just did above. IRC servers like to port scan you before you are allowed to complete a connection to make sure you are not an open proxy being used to do bad things on the IRC server. These port scans fill up logs and hide the real activity that you really want to see in a log with all of this portscanning. Yes, per above, it would be nice to have the time to play the cat & mouse game of handling each one of these "authorized" port scans in a more elegant manner but I don''t have the time for that. I''m jealous of anyone that does. b. ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Tom Eastep
2013-Oct-23 15:22 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 10/23/2013 6:44 AM, Brian J. Murrell wrote:> > I just did above. IRC servers like to port scan you before you are > allowed to complete a connection to make sure you are not an open proxy > being used to do bad things on the IRC server. > > These port scans fill up logs and hide the real activity that you really > want to see in a log with all of this portscanning.One way to approach that is to use per-IP log rate limiting; I have: LOGLIMIT="s:5/min" That way, a typical port scan generates no more than 5 log messages. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Thomas D.
2013-Oct-24 02:57 UTC
Re: correct mechanism to blacklist/drop only incoming connections
Hi, Brian J. Murrell wrote:>> Have a look at the deprecated BLACKLISTNEWONLY > > I''m already using this with =No. > >> or the new BLACKLIST option: >> http://www.shorewall.net/manpages/shorewall.conf.html > > Probably not available in the 4.4.26.1 that I''m using on Ubuntu LTS. > Yeah: > > Added in Shorewall 4.5.13 to replace the BLACKLISTNEWONLY option > below.I guess you were aware about the pro/cons when choosing an LTS release, weren''t you? ;) However, it should be easy to backport a current shorewall release (you could use the official SRC package from testing/SID and re-build it against your LTS version). For me, backporting a package is much more easier than fighting with an outdated version knowing that all your problems are addressed in a current version.>> If a legitimate source will get blacklisted while doing an >> authorized port scan, your portscan rules are way too sensitive. > > Yes. Ideally, handling the portscanning that one (however > reluctantly) agrees to when one connects, say, to an IRC server as > "authorized portscanning" in opposition to unauthorized port scanning > is the way to go, however I don''t want to have to play the cat & > mouse game of deciding who is authorized to do port scans because of > what my users are doing and who is just rattling door knobs randomly. > I have better things to do with my time.With "too sensitive" I meant something like: 1: Source A can try to connect to port 80 at 13:00:00. 2: Source A can try to connect to port 443 at 13:00:01. 3: Source A can try to connect to port 8080 at 13:00:02. 4: But source A cannot try to connect to port 3128 at 13:00:03 anymore, because source A was blacklisted at 13:00:02 due to your portscan rule, which only allows 3 unsuccessful scans within 5 seconds. => 3 unsuccessful connections within 5 seconds are too sensitive (The values are just examples) If you have to deal with this kind of legitimate network traffic, you cannot use such a sensitive rule (or you have to whitelist). It is the same like supporting FTP: Can you control client''s FTP settings? No? So you need to allow a lot of ports because you don''t know which ports your users will use. You could limit the risk if you would maintain a list of allowed FTP addresses. It''s your choice. But you cannot be restrictive, be user-friendly (=not limiting the local ports) and keeping the administrative overhead low (=not maintaining a list of allowed FTP servers) all at the same time. And to be honest, I don''t consider port scans as important. Furthermore I don''t think I gain much information from denied traffic at all (but I am still logging denied traffic, but using a limit like Tom). Money quote:> Enabling logging of "allow" actions gives you visibility into all > traffic into the environment. This is especially important since most > threats target open ports rather than closed.Most important for me to know: What *is* normal traffic in *my* environment? With this knowledge I am able to define trigger and can set alerts to get notified when something unusual will happen. => From my point of view you are doing it wrong if you care so much about typical background noise like port scans and looking for a solution to blacklist one way, but want to bypass blacklist when you start the communication. But don''t get me wrong. I am just sharing my experience with you and I am not saying that I am right :) That''s why I asked for another use case to learn something new (but your IRC example doesn''t convince me, sorry).. -Thomas ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
Brian J. Murrell
2013-Oct-24 13:56 UTC
Re: correct mechanism to blacklist/drop only incoming connections
On 13-10-23 10:57 PM, Thomas D. wrote:> Hi,Hi,> I guess you were aware about the pro/cons when choosing an LTS release, > weren''t you? ;)Of course.> However, it should be easy to backport a current shorewall release (you > could use the official SRC package from testing/SID and re-build it > against your LTS version).Yeah, the building of the packages is not the hard part. It''s deciding to get on that self-maintenance treadmill and more importantly bringing my current configuration up to the level needed by the newer releases. I actually did start down this road the other day but I didn''t have the time to research why my current configuration was causing shorewall to barf.> For me, backporting a package is much more easier than fighting with an > outdated version knowing that all your problems are addressed in a > current version.Sure, if your current configuration "just works".> With "too sensitive" I meant something like: > > 1: Source A can try to connect to port 80 at 13:00:00. > > 2: Source A can try to connect to port 443 at 13:00:01. > > 3: Source A can try to connect to port 8080 at 13:00:02. > > 4: But source A cannot try to connect to port 3128 at 13:00:03 anymore, > because source A was blacklisted at 13:00:02 due to your portscan > rule, which only allows 3 unsuccessful scans within 5 seconds. > > => 3 unsuccessful connections within 5 seconds are too sensitiveI disagree. The time is actually quite irrelevant. It''s the activity. If you want to portscan, welcome to my blacklist. As much as you have a coerced agreement from me to authorize a portscan because I connected to one of your machines, I have an agreement here that if you portscan me, I blacklist you.> If you have to deal with this kind of legitimate"legitimate" is a very grey area here.> network traffic, you > cannot use such a sensitive rule (or you have to whitelist).> And to be honest, I don''t consider port scans as important.They are important in as much as they fill the logs with cruft and bury more interesting stuff.> Furthermore > I don''t think I gain much information from denied traffic at all (but I > am still logging denied traffic, but using a limit like Tom). Money quote: > >> Enabling logging of "allow" actions gives you visibility into all >> traffic into the environment. This is especially important since most >> threats target open ports rather than closed.That is an interesting perspective. b. ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk