Beta 5 is now available for testing.
Problems corrected since Beta 4:
1) Previously, NFACCT accounting rules generated iptables rules with
the matches in the incorrect order. That caused the counters to be
incremented before all of the matches had been checked. Now, the
counter in an NFACCT rule is incremented only if all of the other
matches have been successful.
2) A number of ipset-related modules were incorrectly included in
/usr/share/shorewall/helpers. Those entries have now been removed.
New/modified features since Beta 4:
1) It is now possible to specify HELPERS=none in
/etc/shorewall[6]/shorewall[6].conf.
This setting has two consequences:
a) All of the *_HELPER capabilities are set to off.
b) No probing of helpers is performed, thus eliminating "xt_CT: No
such helper XXX" warnings when the compiler is probing the
system for capabilities.
2) It is now possible to specify multiple nfacct objects in an NFACCT
accounting rule. Where previously, the following rules were given:
SECTION INPUT
NFACCT(all)
NFACCT(all_in)
SECTION OUTPUT
NFACCT(all)
NFACCT(all_out)
SECTION FORWARD
NFACCT(all)
NFACCT(all_fwd)
It is now possible to do the same thing as follows:
SECTION INPUT
NFACCT(all,all_in)
SECTION OUTPUT
NFACCT(all,all_out)
SECTION FORWARD
NFACCT(all,all_fwd)
6) It is now possible to increment an nfacct counter when a packet
matches an ipset. To do that, simply include the counter object''s
name in parentheses after the ipset specification.
Examples:
a) Increment the mysetcounter nfacct object when a packet''s source
matches myset.
+myset[src](mysetcounter)
b) Increment the mysetcounter1 and mysetcounter2 nfacct objects
when a packet''s sourcematches myset.
+myset[src](mysetcounter1,mysetcounter2)
Thank you for testing,
-Tom
--
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
Tom Eastep wrote:> 2) It is now possible to specify multiple nfacct objects in an NFACCT > accounting rule. Where previously, the following rules were given: > > SECTION INPUT > NFACCT(all) > NFACCT(all_in) > > SECTION OUTPUT > NFACCT(all) > NFACCT(all_out) > > SECTION FORWARD > NFACCT(all) > NFACCT(all_fwd) > > It is now possible to do the same thing as follows: > > SECTION INPUT > NFACCT(all,all_in) > > SECTION OUTPUT > NFACCT(all,all_out) > > SECTION FORWARD > NFACCT(all,all_fwd) > > 6) It is now possible to increment an nfacct counter when a packet > matches an ipset. To do that, simply include the counter object''s > name in parentheses after the ipset specification. > > Examples: > > a) Increment the mysetcounter nfacct object when a packet''s source > matches myset. > > +myset[src](mysetcounter) > > b) Increment the mysetcounter1 and mysetcounter2 nfacct objects > when a packet''s sourcematches myset. > > +myset[src](mysetcounter1,mysetcounter2) >I am not sure about the implementation of last two items above. The example I gave yesterday with using ipsets was just that - an example (to illustrate my point). I could have used/added anything else, like: SECTION INPUT NFACCT(all) \ NFACCT(marked) - - - - - 12 \ NFACCT(admin) - - - - - - root \ NFACCT(web) - +web[src,src] The above, if properly "combined" (and, of course, assuming that the "\" symbol activates it) could all be implemented with a single rule: -A accountin -m nfacct --nfacct-name all \ -m mark --mark 0xc -m nfacct --nfacct-name marked \ -m owner --uid-owner 0 -m nfacct --nfacct-name admin \ -m set --match-set web src,src -m nfacct --nfacct-name web If implementing this isn''t possible or very difficult to do (at least for now), I have another possible alternative - implement INLINE in "accounting". That way, I could design the rules myself the way I want it. This though, may require a bit more than the INLINE implemented in rules/blrules. What shorewall currently does when it encounters "NFACCT" in "accounting" is that in addition to producing the nfacct match, it also adds a bunch of statements to create the nfacct object itself using the nfacct executable. To facilitate INLINE in "accounting" (if you are inclined to go that route), you can define a specific keyword (say "NFACCT(obj_name)"), which, if encountered after the ";" symbol, adds the appropriate nfacct match ("-m nfacct --nfacct-name obj_name") at the specified position, as well as adding the appropriate statement for creating the accounting object itself ("nfacct add obj_name" for example). So, using the above example, if you decide to go the INLINE route, in order to produce the same iptables rule as above, all what''s needed is this: SECTION INPUT INLINE ; NFACCT(all) -m mark --mark 0xc NFACCT(marked) -m owner --uid-owner 0 NFACCT(admin) -m set --match-set web src,src NFACCT(web) ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 4/19/13 2:57 PM, "Dash Four" <mr.dash.four@googlemail.com> wrote:> > >Tom Eastep wrote: >> 2) It is now possible to specify multiple nfacct objects in an NFACCT >> accounting rule. Where previously, the following rules were given: >> >> SECTION INPUT >> NFACCT(all) >> NFACCT(all_in) >> >> SECTION OUTPUT >> NFACCT(all) >> NFACCT(all_out) >> >> SECTION FORWARD >> NFACCT(all) >> NFACCT(all_fwd) >> >> It is now possible to do the same thing as follows: >> >> SECTION INPUT >> NFACCT(all,all_in) >> >> SECTION OUTPUT >> NFACCT(all,all_out) >> >> SECTION FORWARD >> NFACCT(all,all_fwd) >> >> 6) It is now possible to increment an nfacct counter when a packet >> matches an ipset. To do that, simply include the counter object''s >> name in parentheses after the ipset specification. >> >> Examples: >> >> a) Increment the mysetcounter nfacct object when a packet''s source >> matches myset. >> >> +myset[src](mysetcounter) >> >> b) Increment the mysetcounter1 and mysetcounter2 nfacct objects >> when a packet''s sourcematches myset. >> >> +myset[src](mysetcounter1,mysetcounter2) >> >I am not sure about the implementation of last two items above. The >example I gave yesterday with using ipsets was just that - an example >(to illustrate my point). I could have used/added anything else, like: > >SECTION INPUT >NFACCT(all) \ >NFACCT(marked) - - - - - 12 \ >NFACCT(admin) - - - - - - root \ >NFACCT(web) - +web[src,src] > >The above, if properly "combined" (and, of course, assuming that the "\" >symbol activates it) could all be implemented with a single rule: > >-A accountin -m nfacct --nfacct-name all \ > -m mark --mark 0xc -m nfacct --nfacct-name marked \ > -m owner --uid-owner 0 -m nfacct --nfacct-name admin \ > -m set --match-set web src,src -m nfacct --nfacct-name webI''ll never implement that.> >If implementing this isn''t possible or very difficult to do (at least >for now), I have another possible alternative - implement INLINE in >"accounting".That I can do. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/19/2013 12:17 PM, Tom Eastep wrote:> Beta 5 is now available for testing. > > Problems corrected since Beta 4: > > 1) Previously, NFACCT accounting rules generated iptables rules with > the matches in the incorrect order. That caused the counters to be > incremented before all of the matches had been checked. Now, the > counter in an NFACCT rule is incremented only if all of the other > matches have been successful. > > 2) A number of ipset-related modules were incorrectly included in > /usr/share/shorewall/helpers. Those entries have now been removed. > > New/modified features since Beta 4: > > 1) It is now possible to specify HELPERS=none in > /etc/shorewall[6]/shorewall[6].conf. > > This setting has two consequences: > > a) All of the *_HELPER capabilities are set to off. > b) No probing of helpers is performed, thus eliminating "xt_CT: No > such helper XXX" warnings when the compiler is probing the > system for capabilities. >I seemed to have had an SCM screwup that resulted in a bad Config.pm being included in the Beta. Patch attached. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
>> SECTION INPUT >> NFACCT(all) \ >> NFACCT(marked) - - - - - 12 \ >> NFACCT(admin) - - - - - - root \ >> NFACCT(web) - +web[src,src] >> >> The above, if properly "combined" (and, of course, assuming that the "\" >> symbol activates it) could all be implemented with a single rule: >> >> -A accountin -m nfacct --nfacct-name all \ >> -m mark --mark 0xc -m nfacct --nfacct-name marked \ >> -m owner --uid-owner 0 -m nfacct --nfacct-name admin \ >> -m set --match-set web src,src -m nfacct --nfacct-name web >> > > I''ll never implement that. >It isn''t easy, I know.>> If implementing this isn''t possible or very difficult to do (at least >> for now), I have another possible alternative - implement INLINE in >> "accounting". >> > > That I can do. >I thought it might be a bit easier than the "\" symbol proposition. It will give me more freedom too. One additional question regarding chains: The man page says that regardless of whether I use SECTION or not, I can always create a custom chain. So, in order to create a "custom" sub-chain in the INPUT main chain, is the following the correct set of statements to use: SECTION INPUT eth0_in - eth0 NFACCT(eth0_in) eth0_in The aim is to produce the following set of rules: :eth0_in -A INPUT -i eth0 -j eth0_in -A eth0_in -m nfacct --nfacct-name eth0_in Have I got this right (the end result shown in the iptables rules above is what I am after)? ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
Tom Eastep wrote:> On 04/19/2013 12:17 PM, Tom Eastep wrote: > >> Beta 5 is now available for testing. >> >> Problems corrected since Beta 4: >> >> 1) Previously, NFACCT accounting rules generated iptables rules with >> the matches in the incorrect order. That caused the counters to be >> incremented before all of the matches had been checked. Now, the >> counter in an NFACCT rule is incremented only if all of the other >> matches have been successful. >> >> 2) A number of ipset-related modules were incorrectly included in >> /usr/share/shorewall/helpers. Those entries have now been removed. >> >> New/modified features since Beta 4: >> >> 1) It is now possible to specify HELPERS=none in >> /etc/shorewall[6]/shorewall[6].conf. >> >> This setting has two consequences: >> >> a) All of the *_HELPER capabilities are set to off. >> b) No probing of helpers is performed, thus eliminating "xt_CT: No >> such helper XXX" warnings when the compiler is probing the >> system for capabilities. >> >> > > I seemed to have had an SCM screwup that resulted in a bad Config.pm > being included in the Beta. Patch attached. >Is this the same implementation as the one I already tested with regards to the HELPERS section?> -Tom > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > ------------------------------------------------------------------------ > > _______________________________________________ > Shorewall-devel mailing list > Shorewall-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/shorewall-devel >------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 4/19/13 3:49 PM, "Dash Four" <mr.dash.four@googlemail.com> wrote:> > >Tom Eastep wrote: >> On 04/19/2013 12:17 PM, Tom Eastep wrote: >> >>> Beta 5 is now available for testing. >>> >>> Problems corrected since Beta 4: >>> >>> 1) Previously, NFACCT accounting rules generated iptables rules with >>> the matches in the incorrect order. That caused the counters to be >>> incremented before all of the matches had been checked. Now, the >>> counter in an NFACCT rule is incremented only if all of the other >>> matches have been successful. >>> >>> 2) A number of ipset-related modules were incorrectly included in >>> /usr/share/shorewall/helpers. Those entries have now been removed. >>> >>> New/modified features since Beta 4: >>> >>> 1) It is now possible to specify HELPERS=none in >>> /etc/shorewall[6]/shorewall[6].conf. >>> >>> This setting has two consequences: >>> >>> a) All of the *_HELPER capabilities are set to off. >>> b) No probing of helpers is performed, thus eliminating "xt_CT: No >>> such helper XXX" warnings when the compiler is probing the >>> system for capabilities. >>> >>> >> >> I seemed to have had an SCM screwup that resulted in a bad Config.pm >> being included in the Beta. Patch attached. >> >Is this the same implementation as the one I already tested with regards >to the HELPERS section?Yes. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 4/19/13 3:47 PM, "Dash Four" <mr.dash.four@googlemail.com> wrote:> >>> SECTION INPUT >>> NFACCT(all) \ >>> NFACCT(marked) - - - - - 12 \ >>> NFACCT(admin) - - - - - - root \ >>> NFACCT(web) - +web[src,src] >>> >>> The above, if properly "combined" (and, of course, assuming that the >>>"\" >>> symbol activates it) could all be implemented with a single rule: >>> >>> -A accountin -m nfacct --nfacct-name all \ >>> -m mark --mark 0xc -m nfacct --nfacct-name marked \ >>> -m owner --uid-owner 0 -m nfacct --nfacct-name admin \ >>> -m set --match-set web src,src -m nfacct --nfacct-name web >>> >> >> I''ll never implement that. >> >It isn''t easy, I know. > >>> If implementing this isn''t possible or very difficult to do (at least >>> for now), I have another possible alternative - implement INLINE in >>> "accounting". >>> >> >> That I can do. >> >I thought it might be a bit easier than the "\" symbol proposition. It >will give me more freedom too. > >One additional question regarding chains: The man page says that >regardless of whether I use SECTION or not, I can always create a custom >chain. So, in order to create a "custom" sub-chain in the INPUT main >chain, is the following the correct set of statements to use: > >SECTION INPUT >eth0_in - eth0 >NFACCT(eth0_in) eth0_in > >The aim is to produce the following set of rules: > >:eth0_in >-A INPUT -i eth0 -j eth0_in >-A eth0_in -m nfacct --nfacct-name eth0_in > >Have I got this right (the end result shown in the iptables rules above >is what I am after)?Yes. But the actual set of rules will be: -A INPUT -j accountin -A accounting -I eth0 -j eth0_in -A eth0_in -m nfacct --nfacct-name eth0_in -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 4/19/13 3:47 PM, "Dash Four" <mr.dash.four@googlemail.com> wrote:> >>> SECTION INPUT >>> NFACCT(all) \ >>> NFACCT(marked) - - - - - 12 \ >>> NFACCT(admin) - - - - - - root \ >>> NFACCT(web) - +web[src,src] >>> >>> The above, if properly "combined" (and, of course, assuming that the >>>"\" >>> symbol activates it) could all be implemented with a single rule: >>> >>> -A accountin -m nfacct --nfacct-name all \ >>> -m mark --mark 0xc -m nfacct --nfacct-name marked \ >>> -m owner --uid-owner 0 -m nfacct --nfacct-name admin \ >>> -m set --match-set web src,src -m nfacct --nfacct-name web >>> >> >> I''ll never implement that. >> >It isn''t easy, I know. > >>> If implementing this isn''t possible or very difficult to do (at least >>> for now), I have another possible alternative - implement INLINE in >>> "accounting". >>> >> >> That I can do. >> >I thought it might be a bit easier than the "\" symbol proposition. It >will give me more freedom too.Patch attached. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
> Yes. But the actual set of rules will be: > > -A INPUT -j accountin > -A accounting -I eth0 -j eth0_in > -A eth0_in -m nfacct --nfacct-name eth0_in >Interesting, I think I found another bug: accounting ~~~~~~~~~~ test INPUT test produces ... well, all hell brakes loose - I get over 30 lines of internal shorewall errors... ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
Tom Eastep wrote:>> I thought it might be a bit easier than the "\" symbol proposition. It >> will give me more freedom too. >> > > Patch attached. >OK, I''ll get on it, but before that I need to ask whether you have the "magical" NFACCT keyword for me to use after the ";" symbol (i.e. "INLINE ; NFACCT(all)" to place the nfacct match as well as include nfacct executable statements to create that object)? ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 4/19/13 5:35 PM, "Dash Four" <mr.dash.four@googlemail.com> wrote:> > >Tom Eastep wrote: >>> I thought it might be a bit easier than the "\" symbol proposition. It >>> will give me more freedom too. >>> >> >> Patch attached. >> >OK, I''ll get on it, but before that I need to ask whether you have the >"magical" NFACCT keyword for me to use after the ";" symbol (i.e. >"INLINE ; NFACCT(all)" to place the nfacct match as well as include >nfacct executable statements to create that object)?There is none -- everything after '';'' is up to you. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
Dash Four wrote:> >> Yes. But the actual set of rules will be: >> >> -A INPUT -j accountin >> -A accounting -I eth0 -j eth0_in >> -A eth0_in -m nfacct --nfacct-name eth0_in >> > Interesting, I think I found another bug: > > accounting > ~~~~~~~~~~ > test > INPUT test > > produces ... well, all hell brakes loose - I get over 30 lines of > internal shorewall errors...OK, I was able to reproduce this and find what is causing it - one of the most bizarre errors I''ve seen. Here goes: 1. cd /etc/shorewall 2. shorewall compile firewall (this MUST succeed with no errors!) 3. edit "accounting" and add: firewall INPUT firewall 4. save & exit, then repeat step 2. I am getting this: String found where operator expected at (eval 22) line 45, near "n "$g_timestamp"" (Do you need to predeclare n?) String found where operator expected at (eval 22) line 46, near "echo "${timestamp}$@"" (Do you need to predeclare echo?) Bareword found where operator expected at (eval 22) line 51, near ""$(date +''%b %_d %T'') " echo" (Missing operator before echo?) String found where operator expected at (eval 22) line 51, near "echo "${timestamp}$@"" (Do you need to predeclare echo?) Bareword found where operator expected at (eval 22) line 52, near "$STARTUP_LOG fi" (Missing operator before fi?) String found where operator expected at (eval 22) line 61, near "n "$g_timestamp"" (Do you need to predeclare n?) String found where operator expected at (eval 22) line 62, near "echo "${timestamp}$@"" (Do you need to predeclare echo?) Bareword found where operator expected at (eval 22) line 67, near ""$(date +''%b %_d %T'') " echo" (Missing operator before echo?) String found where operator expected at (eval 22) line 67, near "echo "${timestamp}$@"" (Do you need to predeclare echo?) Bareword found where operator expected at (eval 22) line 68, near "$STARTUP_LOG fi" (Missing operator before fi?) String found where operator expected at (eval 22) line 77, near "n "$g_timestamp"" ERROR: Couldn''t parse /etc/shorewall/firewall: syntax error at (eval 22) line 40, near ") # $* = Message {" Global symbol "$VERBOSITY" requires explicit package name at (eval 22) line 44, <$currentfile> line 1. Global symbol "$LOG_VERBOSITY" requires explicit package name at (eval 22) line 49, <$currentfile> line 1. syntax error at (eval 22) line 51, near ""$(date +''%b %_d %T'') " echo " Global symbol "$STARTUP_LOG" requires explicit package name at (eval 22) line 52, <$currentfile> line 1. Global symbol "$VERBOSITY" requires explicit package name at (eval 22) line 60, <$currentfile> line 1. Global symbol "$LOG_VERBOSITY" requires explicit package name at (eval 22) line 65, <$currentfile> line 1. syntax error at (eval 22) line 67, near ""$(date +''%b %_d %T'') " echo " Global symbol "$STARTUP_LOG" requires explicit package name at (eval 22) line 68, <$currentfile> line 1. Global symbol "$VERBOSITY" requires explicit package name at (eval 22) line 76, <$currentfile> line 1. (eval 22) has too many errors. /etc/shorewall/accounting (line 1) ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
>> OK, I''ll get on it, but before that I need to ask whether you have the >> "magical" NFACCT keyword for me to use after the ";" symbol (i.e. >> "INLINE ; NFACCT(all)" to place the nfacct match as well as include >> nfacct executable statements to create that object)? >> > > There is none -- everything after '';'' is up to you. >Damn! Is there a chance to add this or implement something similar? I guess if I could include custom actions able to generate output (a string) which is then used to create that part of the rule, then that should work. Something like: action.test ~~~~~~~~~~~ [execute "nfacct add" to create $1 object] RETURN "-m nfacct --nfacct-name $1" and then accounting ~~~~~~~~~~ SECTION INPUT INLINE ; test(all) to generate -A accountin -m nfacct --nfacct-name all as well as execute "nfacct add all" as part of executing the action body of "test". Anyway, found a bug: accounting ~~~~~~~~~~ SECTION INPUT INLINE - +dmz-net ; -m nfacct --nfacct-name test produces -A accountin -m nfacct --nfacct-name test-m set --match-set dmz-net src (note the absence of space character between "test" and "-m"). ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 4/19/13 6:48 PM, "Dash Four" <mr.dash.four@googlemail.com> wrote:> >>> OK, I''ll get on it, but before that I need to ask whether you have the >>> "magical" NFACCT keyword for me to use after the ";" symbol (i.e. >>> "INLINE ; NFACCT(all)" to place the nfacct match as well as include >>> nfacct executable statements to create that object)? >>> >> >> There is none -- everything after '';'' is up to you. >> >Damn! Is there a chance to add this or implement something similar?No. INLINE is a hack to allow raw ip[6]tables commands to be integrated during the compile phase. But it are just that -- raw. It is integrated with the optimizer but that''s about it. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 4/19/13 6:53 PM, "Tom Eastep" <teastep@shorewall.net> wrote:> >No. INLINE is a hack to allow raw ip[6]tables commands to be integrated >during the compile phase. But it are just that -- raw.That breaks new grammatical ground :-) -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/20/2013 12:44 PM, Tom Eastep wrote:> On 4/19/13 6:53 PM, "Tom Eastep" <teastep@shorewall.net> wrote: > >> No. INLINE is a hack to allow raw ip[6]tables commands to be integrated >> during the compile phase. But it are just that -- raw. > That breaks new grammatical ground :-)I thinking your grammar are just fine. ;-) ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/19/2013 06:53 PM, Tom Eastep wrote:> On 4/19/13 6:48 PM, "Dash Four" <mr.dash.four@googlemail.com> wrote: > >> >>>> OK, I''ll get on it, but before that I need to ask whether you have the >>>> "magical" NFACCT keyword for me to use after the ";" symbol (i.e. >>>> "INLINE ; NFACCT(all)" to place the nfacct match as well as include >>>> nfacct executable statements to create that object)? >>>> >>> >>> There is none -- everything after '';'' is up to you. >>> >> Damn! Is there a chance to add this or implement something similar? > > No. INLINE is a hack to allow raw ip[6]tables commands to be integrated > during the compile phase. But it are just that -- raw. It is integrated > with the optimizer but that''s about it. >It is a simple change, however, to automatically record nfacct object names when transforming each rule into the internal form. Will be in the next version. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/20/2013 02:42 AM, Paul Gear wrote:> On 04/20/2013 12:44 PM, Tom Eastep wrote: >> On 4/19/13 6:53 PM, "Tom Eastep" <teastep@shorewall.net> wrote: >> >>> No. INLINE is a hack to allow raw ip[6]tables commands to be integrated >>> during the compile phase. But it are just that -- raw. >> That breaks new grammatical ground :-) > > I thinking your grammar are just fine. ;-) >:-) -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
Paul Gear wrote:> On 04/20/2013 12:44 PM, Tom Eastep wrote: > >> On 4/19/13 6:53 PM, "Tom Eastep" <teastep@shorewall.net> wrote: >> >> >>> No. INLINE is a hack to allow raw ip[6]tables commands to be integrated >>> during the compile phase. But it are just that -- raw. >>> >> That breaks new grammatical ground :-) >> > > I thinking your grammar are just fine. ;-) >It is not the first such instance either (from "man shorewall-accounting" page): "Causes a jump to that chain to be added to the chain specified in the CHAIN column." - care to "translate" Tom? ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
Tom Eastep wrote:> On 04/19/2013 06:53 PM, Tom Eastep wrote: > >> On 4/19/13 6:48 PM, "Dash Four" <mr.dash.four@googlemail.com> wrote: >> >> >>>>> OK, I''ll get on it, but before that I need to ask whether you have the >>>>> "magical" NFACCT keyword for me to use after the ";" symbol (i.e. >>>>> "INLINE ; NFACCT(all)" to place the nfacct match as well as include >>>>> nfacct executable statements to create that object)? >>>>> >>>>> >>>> There is none -- everything after '';'' is up to you. >>>> >>>> >>> Damn! Is there a chance to add this or implement something similar? >>> >> No. INLINE is a hack to allow raw ip[6]tables commands to be integrated >> during the compile phase. But it are just that -- raw. It is integrated >> with the optimizer but that''s about it. >> >> > > It is a simple change, however, to automatically record nfacct object > names when transforming each rule into the internal form. Will be in the > next version. >Thanks! Would that be done "internally" or via "fancy" custom-defined action? ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 4/20/13 8:58 AM, "Dash Four" <mr.dash.four@googlemail.com> wrote:> > >Tom Eastep wrote: >> On 04/19/2013 06:53 PM, Tom Eastep wrote: >> >>> On 4/19/13 6:48 PM, "Dash Four" <mr.dash.four@googlemail.com> wrote: >>> >>> >>>>>> OK, I''ll get on it, but before that I need to ask whether you have >>>>>>the >>>>>> "magical" NFACCT keyword for me to use after the ";" symbol (i.e. >>>>>> "INLINE ; NFACCT(all)" to place the nfacct match as well as include >>>>>> nfacct executable statements to create that object)? >>>>>> >>>>>> >>>>> There is none -- everything after '';'' is up to you. >>>>> >>>>> >>>> Damn! Is there a chance to add this or implement something similar? >>>> >>> No. INLINE is a hack to allow raw ip[6]tables commands to be integrated >>> during the compile phase. But it are just that -- raw. It is integrated >>> with the optimizer but that''s about it. >>> >>> >> >> It is a simple change, however, to automatically record nfacct object >> names when transforming each rule into the internal form. Will be in the >> next version. >> >Thanks! Would that be done "internally" or via "fancy" custom-defined >action?Internally. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 4/20/13 8:57 AM, "Dash Four" <mr.dash.four@googlemail.com> wrote:> > >Paul Gear wrote: >> On 04/20/2013 12:44 PM, Tom Eastep wrote: >> >>> On 4/19/13 6:53 PM, "Tom Eastep" <teastep@shorewall.net> wrote: >>> >>> >>>> No. INLINE is a hack to allow raw ip[6]tables commands to be >>>>integrated >>>> during the compile phase. But it are just that -- raw. >>>> >>> That breaks new grammatical ground :-) >>> >> >> I thinking your grammar are just fine. ;-) >> >It is not the first such instance either (from "man >shorewall-accounting" page): "Causes a jump to that chain to be added to >the chain specified in the CHAIN column." - care to "translate" Tom?When a chain (call it chain2) is specified in the ACTION column: 1) The chain is created. 2) If a chain is specified in the CHAIN column (call it chain1), then a jump from chain1 to chain2 is generated. 2) If no chain is specified in the CHAIN column, then a jump from the default chain (based on SECTION) to chain2 is generated. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
Tom Eastep wrote:> On 4/20/13 8:57 AM, "Dash Four" <mr.dash.four@googlemail.com> wrote: > > >> Paul Gear wrote: >> >>> On 04/20/2013 12:44 PM, Tom Eastep wrote: >>> >>> >>>> On 4/19/13 6:53 PM, "Tom Eastep" <teastep@shorewall.net> wrote: >>>> >>>> >>>> >>>>> No. INLINE is a hack to allow raw ip[6]tables commands to be >>>>> integrated >>>>> during the compile phase. But it are just that -- raw. >>>>> >>>>> >>>> That breaks new grammatical ground :-) >>>> >>>> >>> I thinking your grammar are just fine. ;-) >>> >>> >> It is not the first such instance either (from "man >> shorewall-accounting" page): "Causes a jump to that chain to be added to >> the chain specified in the CHAIN column." - care to "translate" Tom? >> > > When a chain (call it chain2) is specified in the ACTION column: > > 1) The chain is created. > 2) If a chain is specified in the CHAIN column (call it chain1), then a > jump from chain1 to chain2 is generated. > 2) If no chain is specified in the CHAIN column, then a jump from the > default chain (based on SECTION) to chain2 is generated. >The last item on your list is "3)" I take it? :-) Got it now. Maybe you can include this as part of the "chain:COUNT|JUMP" explanation in that man page. So, to if I want to "mimic" what shorewall currently does in rules and create a "net2dmz" accounting zone, the following needs to be done (assuming eth0 serves the "net" zone, while eth1 - the "dmz"): SECTION FORWARD dmz_fwd - - eth1 net2dmz dmz_fwd eth0 That should create the following iptables rules: :accountfwd :dmz_fwd :net2dmz -A FORWARD -j accountfwd -A accountfwd -i eth1 -j dmz_fwd -A dmz_fwd -o eth0 -j net2dmz Correct? ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/20/2013 09:28 AM, Dash Four wrote:> > > Tom Eastep wrote: >> On 4/20/13 8:57 AM, "Dash Four" <mr.dash.four@googlemail.com> wrote: >> >> >>> Paul Gear wrote: >>> >>>> On 04/20/2013 12:44 PM, Tom Eastep wrote: >>>> >>>> >>>>> On 4/19/13 6:53 PM, "Tom Eastep" <teastep@shorewall.net> wrote: >>>>> >>>>> >>>>> >>>>>> No. INLINE is a hack to allow raw ip[6]tables commands to be >>>>>> integrated >>>>>> during the compile phase. But it are just that -- raw. >>>>>> >>>>>> >>>>> That breaks new grammatical ground :-) >>>>> >>>>> >>>> I thinking your grammar are just fine. ;-) >>>> >>>> >>> It is not the first such instance either (from "man >>> shorewall-accounting" page): "Causes a jump to that chain to be added to >>> the chain specified in the CHAIN column." - care to "translate" Tom? >>> >> >> When a chain (call it chain2) is specified in the ACTION column: >> >> 1) The chain is created. >> 2) If a chain is specified in the CHAIN column (call it chain1), then a >> jump from chain1 to chain2 is generated. >> 2) If no chain is specified in the CHAIN column, then a jump from the >> default chain (based on SECTION) to chain2 is generated. >> > The last item on your list is "3)" I take it? :-) > > Got it now. Maybe you can include this as part of the "chain:COUNT|JUMP" > explanation in that man page. So, to if I want to "mimic" what shorewall > currently does in rules and create a "net2dmz" accounting zone, the > following needs to be done (assuming eth0 serves the "net" zone, while > eth1 - the "dmz"): > > SECTION FORWARD > dmz_fwd - - eth1 > net2dmz dmz_fwd eth0 > > That should create the following iptables rules: > > :accountfwd > :dmz_fwd > :net2dmz > -A FORWARD -j accountfwd > -A accountfwd -i eth1 -j dmz_fwd > -A dmz_fwd -o eth0 -j net2dmz > > Correct?[teastep@foobar64 two-interfaces]$ cat accounting # # Shorewall version 4 - Accounting File # # For information about entries in this file, type "man shorewall-accounting" # # Please see http://shorewall.net/Accounting.html for examples and # additional information about how to use this file. # ################################################################################################################# #ACTION CHAIN SOURCE DESTINATION PROTO DEST SOURCE USER/ MARK IPSEC # PORT(S) PORT(S) GROUP SECTION FORWARD dmz_fwd - - eth1 net2dmz dmz_fwd eth0 [teastep@foobar64 two-interfaces]$ shorewall check -r . Checking... Processing /home/teastep/two-interfaces/shorewall.conf... Checking /home/teastep/two-interfaces/zones... ... Optimizing Ruleset... cat << __EOF__ >&3 # # Generated by Shorewall 4.5.15 - Sat Apr 20 09:40:15 2013 # ... *filter :INPUT DROP [0:0] :FORWARD DROP [0:0] :OUTPUT DROP [0:0] :Broadcast - [0:0] :Drop - [0:0] :Reject - [0:0] :accountfwd - [0:0] ... :net2dmz - [0:0] ... -A FORWARD -j accountfwd ... -A accountfwd -o eth1 -j dmz_fwd -A dmz_fwd -i eth0 -j net2dmz ... Shorewall configuration verified [teastep@foobar64 two-interfaces]$ So you reversed the SOURCE/DEST interfaces. Also note though that if you actually have a zone named ''dmz'' and if ACCOUNTING_TABLE=filter, your example won''t work because there is a chain name collision on ''net22dmz''. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
>> Correct? >> > > [teastep@foobar64 two-interfaces]$ cat accounting > # > # Shorewall version 4 - Accounting File > # > # For information about entries in this file, type "man > shorewall-accounting" > # > # Please see http://shorewall.net/Accounting.html for examples and > # additional information about how to use this file. > # > ################################################################################################################# > #ACTION CHAIN SOURCE DESTINATION PROTO DEST > SOURCE USER/ MARK IPSEC > # PORT(S) PORT(S) GROUP > SECTION FORWARD > dmz_fwd - - eth1 > net2dmz dmz_fwd eth0 > [teastep@foobar64 two-interfaces]$ shorewall check -r . > Checking... > Processing /home/teastep/two-interfaces/shorewall.conf... > Checking /home/teastep/two-interfaces/zones... > ... > Optimizing Ruleset... > > cat << __EOF__ >&3 > # > # Generated by Shorewall 4.5.15 - Sat Apr 20 09:40:15 2013 > # > ... > > *filter > :INPUT DROP [0:0] > :FORWARD DROP [0:0] > :OUTPUT DROP [0:0] > :Broadcast - [0:0] > :Drop - [0:0] > :Reject - [0:0] > :accountfwd - [0:0] > ... > :net2dmz - [0:0] > ... > -A FORWARD -j accountfwd > ... > -A accountfwd -o eth1 -j dmz_fwd > -A dmz_fwd -i eth0 -j net2dmz > ... > Shorewall configuration verified > [teastep@foobar64 two-interfaces]$ > > So you reversed the SOURCE/DEST interfaces. >Indeed - I realised that as soon as I ran "shorewall compile" and looked at the produced output. Thanks Tom.> Also note though that if you actually have a zone named ''dmz'' and if > ACCOUNTING_TABLE=filter, your example won''t work because there is a > chain name collision on ''net22dmz''. >Yep, that''s why I use "mangle" instead. Something else you might wish to consider for future implementation: about 80% of my accounting rules will mimic what I have in "rules" (both in terms of chain structure as well as iptables rules/matches), so I am thinking of what could be the best way to "attach" an accounting object to the rules I am interested in, "cloning" the chain structure as well. In other words, if I have, let''s say, a separate column in "rules" for the name of the accounting object to use, then shorewall could then recreate that same set of matches I used in that "rules" statement to attach the nfacct object I specified, also mimicking the chain structure as well. For example: rules ~~~~~ SECTION NEW ACCEPT net $FW:+web-ports [... all other columns ...] web Assuming that "web" was indicated in a new column in rules, then shorewall could attempt to create the same set of matches I used in that rule (ignoring the connection state, of course!), as well as the existing chain structure, and use it to create an accounting object called "web". That would save an enormous amount of work, as well as maintenance (having to sync "rules" with "accounting"). Thoughts? ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/20/2013 10:16 AM, Dash Four wrote:> >>> Correct? >>> >> >> [teastep@foobar64 two-interfaces]$ cat accounting >> # >> # Shorewall version 4 - Accounting File >> # >> # For information about entries in this file, type "man >> shorewall-accounting" >> # >> # Please see http://shorewall.net/Accounting.html for examples and >> # additional information about how to use this file. >> # >> ################################################################################################################# >> #ACTION CHAIN SOURCE DESTINATION PROTO DEST >> SOURCE USER/ MARK IPSEC >> # PORT(S) PORT(S) GROUP >> SECTION FORWARD >> dmz_fwd - - eth1 >> net2dmz dmz_fwd eth0 >> [teastep@foobar64 two-interfaces]$ shorewall check -r . >> Checking... >> Processing /home/teastep/two-interfaces/shorewall.conf... >> Checking /home/teastep/two-interfaces/zones... >> ... >> Optimizing Ruleset... >> >> cat << __EOF__ >&3 >> # >> # Generated by Shorewall 4.5.15 - Sat Apr 20 09:40:15 2013 >> # >> ... >> >> *filter >> :INPUT DROP [0:0] >> :FORWARD DROP [0:0] >> :OUTPUT DROP [0:0] >> :Broadcast - [0:0] >> :Drop - [0:0] >> :Reject - [0:0] >> :accountfwd - [0:0] >> ... >> :net2dmz - [0:0] >> ... >> -A FORWARD -j accountfwd >> ... >> -A accountfwd -o eth1 -j dmz_fwd >> -A dmz_fwd -i eth0 -j net2dmz >> ... >> Shorewall configuration verified >> [teastep@foobar64 two-interfaces]$ >> >> So you reversed the SOURCE/DEST interfaces. >> > Indeed - I realised that as soon as I ran "shorewall compile" and looked > at the produced output. Thanks Tom. > >> Also note though that if you actually have a zone named ''dmz'' and if >> ACCOUNTING_TABLE=filter, your example won''t work because there is a >> chain name collision on ''net22dmz''. >> > Yep, that''s why I use "mangle" instead. > > Something else you might wish to consider for future implementation: > about 80% of my accounting rules will mimic what I have in "rules" (both > in terms of chain structure as well as iptables rules/matches), so I am > thinking of what could be the best way to "attach" an accounting object > to the rules I am interested in, "cloning" the chain structure as well. > > In other words, if I have, let''s say, a separate column in "rules" for > the name of the accounting object to use, then shorewall could then > recreate that same set of matches I used in that "rules" statement to > attach the nfacct object I specified, also mimicking the chain structure > as well. For example: > > rules > ~~~~~ > SECTION NEW > ACCEPT net $FW:+web-ports [... all other columns ...] web > > Assuming that "web" was indicated in a new column in rules, then > shorewall could attempt to create the same set of matches I used in that > rule (ignoring the connection state, of course!), as well as the > existing chain structure, and use it to create an accounting object > called "web". That would save an enormous amount of work, as well as > maintenance (having to sync "rules" with "accounting"). Thoughts? >Why does it have to be a separate set of chains? If you are using nfacct, why not just bump the accounting objects in the rules chains? -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
> Why does it have to be a separate set of chains? If you are using > nfacct, why not just bump the accounting objects in the rules chains? >Because of the connection state. Most (if not all) of the rules present in "rules" depend on or are executed only when a certain connection state matches. So if I just include the nfacct object as part of the original rules (in "rules") as you suggest, then I am only going to count packets in the state in which that particular SECTION operates, which is, obviously, not what counts (pun intended). Even if I use SECTION ALL rules, then I have to duplicate (and maintain) stuff there as well. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/20/2013 11:57 AM, Dash Four wrote:> >> Why does it have to be a separate set of chains? If you are using >> nfacct, why not just bump the accounting objects in the rules chains? >> > Because of the connection state. > > Most (if not all) of the rules present in "rules" depend on or are > executed only when a certain connection state matches. So if I just > include the nfacct object as part of the original rules (in "rules") as > you suggest, then I am only going to count packets in the state in which > that particular SECTION operates, which is, obviously, not what counts > (pun intended). Even if I use SECTION ALL rules, then I have to > duplicate (and maintain) stuff there as well.What if we simply add a new NFACCT column to the POLICY file to name the counter you want to use for traffic between the specified pair of zones? The compiler could then insert an nfacct match rules as the first entry in the corresponding rules chain. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/20/2013 12:10 PM, Tom Eastep wrote:> > What if we simply add a new NFACCT column to the POLICY file to name the > counter you want to use for traffic between the specified pair of zones? > The compiler could then insert an nfacct match rules as the first entry > in the corresponding rules chain.I guess that you want finer-grained accounting than that? -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
Tom Eastep wrote:> On 04/20/2013 12:10 PM, Tom Eastep wrote: > >> What if we simply add a new NFACCT column to the POLICY file to name the >> counter you want to use for traffic between the specified pair of zones? >> The compiler could then insert an nfacct match rules as the first entry >> in the corresponding rules chain. >> > > I guess that you want finer-grained accounting than that? >I am not sure I understand you Tom. The idea is very simple, although I admit the implementation of it might be a bit challenging: when I have a bunch of "important" rules in "rules", which I need counted towards traffic, I''d like to simply add an nfacct object name, using the same set of rules specified in a particular "rules" statement to count packets/bytes, but with 2 very important exceptions: 1. The state of that accounting rule must be ignored (so that I always count packets/bytes, regardless of what state that connection is, although using the same constraints - understand matches - I used to construct the original "rules" rule); and 2. That accounting rule needs to satisfy the same chain conditions for which the original "rules" rule was specified. I''ll use the example I gave earlier to better illustrate my point: rules ~~~~~ SECTION NEW #ACTION SOURCE DEST [... other columns ...] NFACCT [...] ACCEPT net $FW:+web-ports [... other columns ...] web The above will produce iptables rule which will only be executed when the state of the connection is NEW. In other words, the above statement will be translated by shorewall to: -A net2fw -m set --match-set web-ports dst What I''d also like to have (which is indicated by the word "web" present in the NFACCT column above) is to have the "same" rule applied in order to count bytes/packets for the "web" nfacct object. In other words, by including "web" in the NFACCT column, to ask shorewall to produce the following rule: -A net2fw -m set --match-set web-ports src -m nfacct --nfacct-name web The above statement *must* use the same chain structure I used in "rules": INPUT -> net2fw (with the possible exception that the chain could be INPUT -> accountin -> net2fw if that is more desirable) so that "net2fw" is traversed only when the "net" zone interface is hit - exactly the same as in "net2fw" in rules. That way, by only specifying the accounting object name in the "new" NFACCT column as part of my "rules", I''ll save myself the horror of having to "clone" (and further maintain) about 80% of my accounting rules, which would be of great benefit, obviously. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
Dash Four wrote:> > Dash Four wrote: >> >>> Yes. But the actual set of rules will be: >>> >>> -A INPUT -j accountin >>> -A accounting -I eth0 -j eth0_in >>> -A eth0_in -m nfacct --nfacct-name eth0_in >>> >> Interesting, I think I found another bug: >> >> accounting >> ~~~~~~~~~~ >> test >> INPUT test >> >> produces ... well, all hell brakes loose - I get over 30 lines of >> internal shorewall errors... > OK, I was able to reproduce this and find what is causing it - one of > the most bizarre errors I''ve seen. Here goes: > 1. cd /etc/shorewall > 2. shorewall compile firewall (this MUST succeed with no errors!) > 3. edit "accounting" and add: > firewall > INPUT firewall > 4. save & exit, then repeat step 2. I am getting this: > > String found where operator expected at (eval 22) line 45, near "n > "$g_timestamp"" > (Do you need to predeclare n?) > String found where operator expected at (eval 22) line 46, near "echo > "${timestamp}$@"" > (Do you need to predeclare echo?) > Bareword found where operator expected at (eval 22) line 51, near > ""$(date +''%b %_d %T'') " > echo" > (Missing operator before echo?) > String found where operator expected at (eval 22) line 51, near "echo > "${timestamp}$@"" > (Do you need to predeclare echo?) > Bareword found where operator expected at (eval 22) line 52, near > "$STARTUP_LOG > fi" > (Missing operator before fi?) > String found where operator expected at (eval 22) line 61, near "n > "$g_timestamp"" > (Do you need to predeclare n?) > String found where operator expected at (eval 22) line 62, near "echo > "${timestamp}$@"" > (Do you need to predeclare echo?) > Bareword found where operator expected at (eval 22) line 67, near > ""$(date +''%b %_d %T'') " > echo" > (Missing operator before echo?) > String found where operator expected at (eval 22) line 67, near "echo > "${timestamp}$@"" > (Do you need to predeclare echo?) > Bareword found where operator expected at (eval 22) line 68, near > "$STARTUP_LOG > fi" > (Missing operator before fi?) > String found where operator expected at (eval 22) line 77, near "n > "$g_timestamp"" > ERROR: Couldn''t parse /etc/shorewall/firewall: syntax error at (eval > 22) line 40, near ") # $* = Message > {" > Global symbol "$VERBOSITY" requires explicit package name at (eval 22) > line 44, <$currentfile> line 1. > Global symbol "$LOG_VERBOSITY" requires explicit package name at (eval > 22) line 49, <$currentfile> line 1. > syntax error at (eval 22) line 51, near ""$(date +''%b %_d %T'') " > echo " > Global symbol "$STARTUP_LOG" requires explicit package name at (eval > 22) line 52, <$currentfile> line 1. > Global symbol "$VERBOSITY" requires explicit package name at (eval 22) > line 60, <$currentfile> line 1. > Global symbol "$LOG_VERBOSITY" requires explicit package name at (eval > 22) line 65, <$currentfile> line 1. > syntax error at (eval 22) line 67, near ""$(date +''%b %_d %T'') " > echo " > Global symbol "$STARTUP_LOG" requires explicit package name at (eval > 22) line 68, <$currentfile> line 1. > Global symbol "$VERBOSITY" requires explicit package name at (eval 22) > line 76, <$currentfile> line 1. > (eval 22) has too many errors. > /etc/shorewall/accounting (line 1)The above error is still present in this Beta! ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/20/2013 05:26 PM, Dash Four wrote:>>> Interesting, I think I found another bug:>> (eval 22) has too many errors. >> /etc/shorewall/accounting (line 1) > The above error is still present in this Beta!And I don''t know what to do about it. There is an ancient precident in Shorewall whereby if there is a file on the CONFIG_PATH with the same name as a chain being created, then that file is considered to be a compile-time extension script. So when you try to create a chain called ''firewall'', the compiler is attempting to run ''firewall'' as a Perl script. I don''t know if anyone has ever taken advantage of that ''feature''... -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/20/2013 05:44 PM, Tom Eastep wrote:> On 04/20/2013 05:26 PM, Dash Four wrote: > >>>> Interesting, I think I found another bug: > >>> (eval 22) has too many errors. >>> /etc/shorewall/accounting (line 1) >> The above error is still present in this Beta! > > And I don''t know what to do about it. There is an ancient precident in > Shorewall whereby if there is a file on the CONFIG_PATH with the same > name as a chain being created, then that file is considered to be a > compile-time extension script. So when you try to create a chain called > ''firewall'', the compiler is attempting to run ''firewall'' as a Perl > script. I don''t know if anyone has ever taken advantage of that ''feature''... >This will be in the next release (probably RC 1): 8) Prior to the availability of BEGIN PERL....END PERL in configuration files, the only way to execute a chain-specific script was to create a script file with the same name as the chain and place it in a directory on the CONFIG_PATH. That facility has the drawback that the compiler will attempt to run a non-script file just because it has the same name as a chain. To disable this facility, a new CHAIN_SCRIPTS option has been added to shorewall[6].conf. The facility is disabled by setting CHAIN_SCRIPTS=No. If not specified or specified as the empty value, CHAIN_SCRIPTS=Yes is assumed for backward compatibility. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 04/20/2013 05:44 PM, Tom Eastep wrote:> On 04/20/2013 05:26 PM, Dash Four wrote: > >>>> Interesting, I think I found another bug: > >>> (eval 22) has too many errors. >>> /etc/shorewall/accounting (line 1) >> The above error is still present in this Beta! > > And I don''t know what to do about it. There is an ancient precident inHmmm -- good spelling too. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
> This will be in the next release (probably RC 1): > > 8) Prior to the availability of BEGIN PERL....END PERL in > configuration files, the only way to execute a chain-specific > script was to create a script file with the same name as the chain > and place it in a directory on the CONFIG_PATH. That facility has > the drawback that the compiler will attempt to run a non-script > file just because it has the same name as a chain. To disable this > facility, a new CHAIN_SCRIPTS option has been added to > shorewall[6].conf. The facility is disabled by setting > CHAIN_SCRIPTS=No. If not specified or specified as the empty value, > CHAIN_SCRIPTS=Yes is assumed for backward compatibility. >Yep, looks reasonable. I''ll probably deactivate this as it won''t be needed. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
Dash Four wrote:> The idea is very simple, although I admit the implementation of it > might be a bit challenging: when I have a bunch of "important" rules > in "rules", which I need counted towards traffic, I''d like to simply > add an nfacct object name, using the same set of rules specified in a > particular "rules" statement to count packets/bytes, but with 2 very > important exceptions: > > 1. The state of that accounting rule must be ignored (so that I always > count packets/bytes, regardless of what state that connection is, > although using the same constraints - understand matches - I used to > construct the original "rules" rule); and > 2. That accounting rule needs to satisfy the same chain conditions for > which the original "rules" rule was specified.That''s not as easy as I thought it would be - the rules in "rules" describe one part of the connection as the opposite part is usually accepted as the cstate is ESTABLISHED. What that means in practice is, if the above is to be implemented, then it has to apply to both side of the connection, reversing the matches and I think that is a bridge too far - I don''t know how it could be done! ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter
On 4/21/13 7:18 PM, "Dash Four" <mr.dash.four@googlemail.com> wrote:> >Dash Four wrote: >> The idea is very simple, although I admit the implementation of it >> might be a bit challenging: when I have a bunch of "important" rules >> in "rules", which I need counted towards traffic, I''d like to simply >> add an nfacct object name, using the same set of rules specified in a >> particular "rules" statement to count packets/bytes, but with 2 very >> important exceptions: >> >> 1. The state of that accounting rule must be ignored (so that I always >> count packets/bytes, regardless of what state that connection is, >> although using the same constraints - understand matches - I used to >> construct the original "rules" rule); and >> 2. That accounting rule needs to satisfy the same chain conditions for >> which the original "rules" rule was specified. >That''s not as easy as I thought it would be - the rules in "rules" >describe one part of the connection as the opposite part is usually >accepted as the cstate is ESTABLISHED. What that means in practice is, >if the above is to be implemented, then it has to apply to both side of >the connection, reversing the matches and I think that is a bridge too >far - I don''t know how it could be done!Yep -- I had already realized that. I''m not sure that I could ever get that right... -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter