Beta 3 is now available for testing.
Problems Corrected since Beta 2
1) Under rare circumstances, optimize level 16 could produce invalid
iptables-restore input which would case start/restart to fail.
New/Changed Features since Beta 2
1) A ULOG ACTION has been added to /etc/shorewall/rules.
2) Within an action body, the variable $0 now expands to the action
chain name (including leading ''%'' if present).
3) ''In-line'' actions are now available. An action is
designated as
in-line within /etc/shorewall[6]/actions; that file has a
new OPTIONS column and specifying ''inline'' in that column
designates the action as in-line.
Normally, actions are expanded into their own chain with a
unique chain being created for each unique invocation (considering
log level, tag and parameters). An in-line actions is expanded
inline within the chain that invokes it. In that sense,
in-line actions are very similar to macros.
In-line actions differ from macros in several ways:
a) A zone may be specified in the SOURCE and DEST columns of a
macro, while zone names are disallowed in these columns within
an in-line action (same as in a regular action).
b) The name of the current chain is available in $0 within the body
of an in-line action (also within a regular action beginning with
Beta 3).
c) In-line actions accept multiple parameters which are available
in$1, $2, etc (as they are in a regular action).
d) PARAM has no special meaning in the body of an in-line action
($1 serves the same purpose in an in-line action).
e) Only FORMAT 2 is available in an in-line action.
f) In-line actions must be defined in
/etc/shorewall[6]/actions. Those files have been extended to
include an OPTIONS column. The only option currently supported
is ''in-line''.
In-line actions differ from normal actions in that:
a) Obviously, they are expanded in-line like a macro rather than
being in their own chain. That means that columns in the
invocation are merged with those in the action body in the same
way as they are in a macro.
b) When AUTOCOMMENT=Yes, each generated rule is commented with the
name of an in-line action.
c) Within an in-line action, ?BEGIN PERL ... ?END PERL does not
have access to the special features available in action a normal
action body.
The order in which the user''s actions file and the actions.std file
are processed has been reversed so that the user''s file is
processed first. This allows overriding the setting of
''inline'' on
the Shorewall standard actions. Beware, however, that some of them
don''t work when inlined and will generate a fatal error message if
you try to inline them:
Broadcast
DropSmurfs
Invalid
NonSyn
RST
TCPFlags
4) In SWITCH columns, the named switch can now be initialized by the
''start'' command (other commands do not change switch
values).
Initialization is accomplished by adding ''=0'' or
''=1'' to the
switch name.
Example (using alternative rule column specification):
#ACTION SOURCE DEST ...
NFLOG all all ; switch=>logall=1
The above will cause the ''logall'' switch
(/proc/net/nf_condition/logall) to be initialized to 1 (on). Note
that netfilter provides no atomic way to define and initialize a
switch so the loading of the ruleset and the initialization of the
switches are distinct operations.
-Tom
--
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
------------------------------------------------------------------------------
Keep yourself connected to Go Parallel:
VERIFY Test and improve your parallel project with help from experts
and peers. http://goparallel.sourceforge.net
> Initialization is accomplished by adding ''=0'' or ''=1'' to the > switch name. > > Example (using alternative rule column specification): > > #ACTION SOURCE DEST ... > NFLOG all all ; switch=>logall=1 >Interesting! When was this ''alternative'' format introduced (I must''ve been half-asleep then since it is the first time I am seeing this)? I also take it this format has been documented in shorewall, right?> Note that netfilter provides no atomic way to define and initialize a > switch so the loading of the ruleset and the initialization of the > switches are distinct operations. >I looked (albeit briefly) in the "condition" kernel module/iptables target sources and that could be easily rectified by allowing an additional value (0 or 1) to be specified when the iptables rule is created. Since Jan Engelhardt is the author (this is part of xtables-addons and not the kernel itself) I may propose this patch to the netfilter devs and see where that takes us. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net
On 11/29/2012 04:44 AM, Mr Dash Four wrote:> >> Initialization is accomplished by adding ''=0'' or ''=1'' to the >> switch name. >> >> Example (using alternative rule column specification): >> >> #ACTION SOURCE DEST ... >> NFLOG all all ; switch=>logall=1 >> > Interesting! When was this ''alternative'' format introduced (I must''ve > been half-asleep then since it is the first time I am seeing this)? I > also take it this format has been documented in shorewall, right?It was introduced in 4.4.24 after a long discussion on this list. This format (along with many other useful tips) is documented at http://www.shorewall.net/configuration_file_basics.htm.> >> Note that netfilter provides no atomic way to define and initialize a >> switch so the loading of the ruleset and the initialization of the >> switches are distinct operations. >> > I looked (albeit briefly) in the "condition" kernel module/iptables > target sources and that could be easily rectified by allowing an > additional value (0 or 1) to be specified when the iptables rule is > created. Since Jan Engelhardt is the author (this is part of > xtables-addons and not the kernel itself) I may propose this patch to > the netfilter devs and see where that takes us.I thought of that also. I suspect, however, that dealing with different initializations in different rules might be tricky there; especially if the rules were in different tables. It would probably be a case of ''first rule wins''. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net
> It was introduced in 4.4.24 after a long discussion on this list. This > format (along with many other useful tips) is documented at > http://www.shorewall.net/configuration_file_basics.htm. >Very useful, thanks.> I thought of that also. I suspect, however, that dealing with different > initializations in different rules might be tricky there; especially if > the rules were in different tables.My gut instinct tells me that the behaviour should be the same when a condition match with the same name is used in different rules - i.e. accept it and silently ignore the default value specified if this particular switch already exists. One example: if I create a rule containing a match called ''foo'', then from the command line I set /proc/net/nf_condition/foo to 1, and then create another rule with the same switch name, the (already set) value of ''foo'' remains intact. In other words: #iptables -N test #iptables -A test -m condition --condition foo #cat /proc/net/nf_condition/foo 0 #echo 1 > /proc/net/nf_condition/foo #cat /proc/net/nf_condition/foo 1 #iptables -A test -m condition --condition foo #cat /proc/net/nf_condition/foo 1 So, what I think should happen is, if I specify something like "iptables -A test -m condition --condition foo 0" and ''foo'' already exists, the default value specified (0 in this case) should be silently ignored, simply because the existing condition implementation in xtables-addons does that implicitly.> It would probably be a case of ''first rule wins''. >Yep, the question is - how is shorewall currently handling this - silently ignore the default value, issue a warning or issue an error? ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net
On 11/29/2012 08:36 AM, Mr Dash Four wrote:> Yep, the question is - how is shorewall currently handling this - > silently ignore the default value, issue a warning or issue an error?It issues an error when a conflicting initialization is detected. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net
> It issues an error when a conflicting initialization is detected. >I see. I just had more time to look over the condition target code (both kernel and iptables) and don''t foresee any problems with introducing this new parameter - I might have a working copy of these changes by the weekend. If that happens, would you be willing to change the way shorewall initialises the switches? If so, you may need to introduce some sort of new/old condition match capability though. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net
On 11/29/2012 10:00 AM, Mr Dash Four wrote:> >> It issues an error when a conflicting initialization is detected. >> > I see. I just had more time to look over the condition target code (both > kernel and iptables) and don''t foresee any problems with introducing > this new parameter - I might have a working copy of these changes by the > weekend. If that happens, would you be willing to change the way > shorewall initialises the switches?Sure. > If so, you may need to introduce> some sort of new/old condition match capability though.Yes. I assume that you will be willing to test the detection of that capability (I can obviously test detection of its absence); it needs to be tested with ''show capabilities'' and in the compiler. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net
On 11/29/2012 10:32 AM, Tom Eastep wrote:> On 11/29/2012 10:00 AM, Mr Dash Four wrote: >> If so, you may need to introduce >> some sort of new/old condition match capability though. > > Yes. I assume that you will be willing to test the detection of that > capability (I can obviously test detection of its absence); it needs to > be tested with ''show capabilities'' and in the compiler. >I just realized that condition match is part of xtables-addons so if you want to send me your patch, I''ll test it with Shorewall. I believe that the xtables parser requires an option per value so the syntax will be something like: condition match options: [!] --condition name Match on boolean value stored in procfs file --condinit value -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net
Tom
In the attached config., when the rules file contains:
okay all all tcp 1
and action.okay contains
ACCEPT {switch=$0=1}
the following messages are produced:
/var/lib/shorewall/.start: 1: cannot create /proc/net/nf_condition/brd22p8:
Directory nonexistent
/var/lib/shorewall/.start: 1: cannot create /proc/net/nf_condition/brd22p7:
Directory nonexistent
/var/lib/shorewall/.start: 1: cannot create /proc/net/nf_condition/p62fw:
Directory nonexistent
/var/lib/shorewall/.start: 1: cannot create /proc/net/nf_condition/dmz2p2:
Directory nonexistent
/var/lib/shorewall/.start: 1: cannot create /proc/net/nf_condition/brd2p2:
Directory nonexistent
/var/lib/shorewall/.start: 1: cannot create /proc/net/nf_condition/dmz2p1:
Directory nonexistent
/var/lib/shorewall/.start: 1: cannot create /proc/net/nf_condition/brd2p1:
Directory nonexistent
Steven.
------------------------------------------------------------------------------
Keep yourself connected to Go Parallel:
VERIFY Test and improve your parallel project with help from experts
and peers. http://goparallel.sourceforge.net
On 11/29/2012 02:02 PM, Steven Jan Springl wrote:> In the attached config., when the rules file contains: > > okay all all tcp 1 > > and action.okay contains > > ACCEPT {switch=$0=1} > > the following messages are produced: > > /var/lib/shorewall/.start: 1: cannot create /proc/net/nf_condition/brd22p8: > Directory nonexistent >Good catch, Steven. Patch attached. Thanks, -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net
On 11/29/2012 02:55 PM, Tom Eastep wrote:> On 11/29/2012 02:02 PM, Steven Jan Springl wrote: > >> In the attached config., when the rules file contains: >> >> okay all all tcp 1 >> >> and action.okay contains >> >> ACCEPT {switch=$0=1}For those reading along, I find the following form to be more readable; it avoids the double ''=''. ACCEPT {switch:$0=1} -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net
On Thursday 29 Nov 2012 22:55:29 Tom Eastep wrote:> On 11/29/2012 02:02 PM, Steven Jan Springl wrote: > > In the attached config., when the rules file contains: > > > > okay all all tcp 1 > > > > and action.okay contains > > > > ACCEPT {switch=$0=1} > > > > the following messages are produced: > > > > /var/lib/shorewall/.start: 1: cannot create > > /proc/net/nf_condition/brd22p8: Directory nonexistent > > Good catch, Steven. > > Patch attached. > > Thanks, > -TomTom The patch doesn''t apply, I get: Hunk #1 FAILED at 7491. Steven. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net
On 11/29/2012 03:31 PM, Steven Jan Springl wrote:> > The patch doesn''t apply, I get: > > Hunk #1 FAILED at 7491. >Sorry - the file changed since Beta 3 -- here is a patch against Beta 3. Thanks Steven, -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net
On 11/30/2012 08:55 AM, Tom Eastep wrote:> On 11/29/2012 02:02 PM, Steven Jan Springl wrote: > >> In the attached config., when the rules file contains: >> >> okay all all tcp 1 >> >> and action.okay contains >> >> ACCEPT {switch=$0=1} >> >> the following messages are produced: >> >> /var/lib/shorewall/.start: 1: cannot create >> /proc/net/nf_condition/brd22p8: >> Directory nonexistent >> > > Good catch, Steven. > > Patch attached. > > Thanks, > -TomSometimes i just sit to ponder the beauteous evils that are Steven''s test cases. I swear he sits there looking at your patches for ways to break the parser. Paul ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net
On 11/29/12 4:03 PM, Tom Eastep wrote:> On 11/29/2012 03:31 PM, Steven Jan Springl wrote: > >> >> The patch doesn''t apply, I get: >> >> Hunk #1 FAILED at 7491. >> > > Sorry - the file changed since Beta 3 -- here is a patch against Beta 3.It was a subtle white-space difference. The patch looked correct to the naked eye. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net
On 11/29/12 4:04 PM, Paul Gear wrote:> > Sometimes i just sit to ponder the beauteous evils that are Steven''s > test cases. I swear he sits there looking at your patches for ways to > break the parser.Yes -- I''ve often suspected that Steven does "white-box" testing :-) -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net
> I just realized that condition match is part of xtables-addons so if you > want to send me your patch, I''ll test it with Shorewall.Yeah, I know - I trashed the entire kernel and iptables sources a couple of days ago and was tearing my hair out when I couldn''t find anything when realised that this is part of the xtables-addons.> I believe that the xtables parser requires an option per value so the > syntax will be something like: > > condition match options: > [!] --condition name Match on boolean value stored in procfs file > --condinit valueEven though I just implemented this, I have some serious misgivings, the main one being inconsistency. Lets take the following example: iptables -t raw -N foo_raw iptables -N foo_filter iptables -t raw -A foo_raw -m condition --condition foo --init 1 iptables -A foo_filter -m condition --condition foo --init 0 Since the "condition" match operates on the first-rule-wins basis, "foo" will have been initialised with a value of "1" (enabled). When executing iptables-save, these values would have been included in the resulting restore file as well, however, the order in which iptables-save operates isn''t guaranteed to be the order in which "foo" has been initialised. Simply put, "foo" is not guaranteed to be "1" after iptables-save/iptables-restore cycle is complete. The same is valid if last-rule-wins is adopted. Even if I show the current value of "foo" when iptables -L is executed (1 in my example above), or, include this same value in iptables-save, this would be, again, inconsistent with what was originally entered. So, it is a heads I lose, tails you win type scenario. As for the Beta3 release, I''ll have a bit more time during the weekend and will give it more thorough examination. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net
On 11/29/12 4:34 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:> >> I just realized that condition match is part of xtables-addons so if >>you >> want to send me your patch, I''ll test it with Shorewall. >Yeah, I know - I trashed the entire kernel and iptables sources a couple >of days ago and was tearing my hair out when I couldn''t find anything >when realised that this is part of the xtables-addons. > >> I believe that the xtables parser requires an option per value so the >> syntax will be something like: >> >> condition match options: >> [!] --condition name Match on boolean value stored in procfs file >> --condinit value >Even though I just implemented this, I have some serious misgivings, the >main one being inconsistency. Lets take the following example: > >iptables -t raw -N foo_raw >iptables -N foo_filter >iptables -t raw -A foo_raw -m condition --condition foo --init 1 >iptables -A foo_filter -m condition --condition foo --init 0 > > >Since the "condition" match operates on the first-rule-wins basis, "foo" >will have been initialised with a value of "1" (enabled). > >When executing iptables-save, these values would have been included in >the resulting restore file as well, however, the order in which >iptables-save operates isn''t guaranteed to be the order in which "foo" >has been initialised. Simply put, "foo" is not guaranteed to be "1" after >iptables-save/iptables-restore cycle is complete. The same is valid if >last-rule-wins is adopted. > >Even if I show the current value of "foo" when iptables -L is executed (1 >in my example above), or, include this same value in iptables-save, this >would be, again, inconsistent with what was originally entered. So, it is >a heads I lose, tails you win type scenario.If you simply use the current value in ''save'', all rules referencing the condition should be consistent unless there is a iptables-restore/"echo > /proc/..." race, right?> >As for the Beta3 release, I''ll have a bit more time during the weekend >and will give it more thorough examination.Thanks, -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net
>> Even though I just implemented this, I have some serious misgivings, the >> main one being inconsistency. Lets take the following example: >> >> iptables -t raw -N foo_raw >> iptables -N foo_filter >> iptables -t raw -A foo_raw -m condition --condition foo --init 1 >> iptables -A foo_filter -m condition --condition foo --init 0 >> >> >> Since the "condition" match operates on the first-rule-wins basis, "foo" >> will have been initialised with a value of "1" (enabled). >> >> When executing iptables-save, these values would have been included in >> the resulting restore file as well, however, the order in which >> iptables-save operates isn''t guaranteed to be the order in which "foo" >> has been initialised. Simply put, "foo" is not guaranteed to be "1" after >> iptables-save/iptables-restore cycle is complete. The same is valid if >> last-rule-wins is adopted. >> >> Even if I show the current value of "foo" when iptables -L is executed (1 >> in my example above), or, include this same value in iptables-save, this >> would be, again, inconsistent with what was originally entered. So, it is >> a heads I lose, tails you win type scenario. > > If you simply use the current value in ''save'', all rules referencing the > condition should be consistent unless there is a iptables-restore/"echo > > /proc/..." race, right?Even if I do that, this will be inconsistent with what was originally entered (see "--init 0" above). Not to mention that when I execute the last statement in my example above and then issue "iptables -L -vn" this will print a value of "1" where I initially entered "0", which is inconsistent to say the least. Besides, the content of a file isn''t guaranteed to be the same unless that file has an explicit lock, so, no, that file content ("1" in our case) isn''t guaranteed to be in iptables-save either. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net
On Friday 30 Nov 2012 00:03:52 Tom Eastep wrote:> On 11/29/2012 03:31 PM, Steven Jan Springl wrote: > > The patch doesn''t apply, I get: > > > > Hunk #1 FAILED at 7491. > > Sorry - the file changed since Beta 3 -- here is a patch against Beta 3. > > Thanks Steven, > -TomTom Confirmed, the patch corrects the issue. Thanks. Steven. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net
On Friday 30 Nov 2012 00:04:15 Paul Gear wrote:> > Sometimes i just sit to ponder the beauteous evils that are Steven''s > test cases. I swear he sits there looking at your patches for ways to > break the parser. >Never, that would be cheating. My main test suite takes over 3 minutes to start on a dedicated 2GHZ processor and generates an iptables-restore with more than 98,000 lines. The real challenge is not to find a bug, but to find what part of my test suit triggered the bug. Steven. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net
On 11/29/12 5:34 PM, "Steven Jan Springl" <steven@springl.ukfsn.org> wrote:>On Friday 30 Nov 2012 00:04:15 Paul Gear wrote: >> >> Sometimes i just sit to ponder the beauteous evils that are Steven''s >> test cases. I swear he sits there looking at your patches for ways to >> break the parser. >> > >Never, that would be cheating. > >My main test suite takes over 3 minutes to start on a dedicated 2GHZ >processor >and generates an iptables-restore with more than 98,000 lines. > >The real challenge is not to find a bug, but to find what part of my test >suit >triggered the bug.And I really appreciate your isolating the problem as you do. I sure makes my job a lot easier. Thanks again Steven, -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net
> c) Within an in-line action, ?BEGIN PERL ... ?END PERL does not > have access to the special features available in action a normal > action body. > > The order in which the user''s actions file and the actions.std file > are processed has been reversed so that the user''s file is > processed first. This allows overriding the setting of ''inline'' on > the Shorewall standard actions. Beware, however, that some of them > don''t work when inlined and will generate a fatal error message if > you try to inline them: > > Broadcast > DropSmurfs > Invalid > NonSyn > RST > TCPFlagsTom, could you clarify this a bit more please? ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 11/30/12 6:44 PM, Mr Dash Four wrote:> >> c) Within an in-line action, ?BEGIN PERL ... ?END PERL does not >> have access to the special features available in action a normal >> action body. >> >> The order in which the user''s actions file and the actions.std file >> are processed has been reversed so that the user''s file is >> processed first. This allows overriding the setting of ''inline'' on >> the Shorewall standard actions. Beware, however, that some of them >> don''t work when inlined and will generate a fatal error message if >> you try to inline them: >> >> Broadcast >> DropSmurfs >> Invalid >> NonSyn >> RST >> TCPFlags > Tom, could you clarify this a bit more please?If you specify ''inline'' for any of these actions in your /etc/shorewall/actions file and then invoke one of them in your configuration, a fatal compilation error will be raised. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
> If you specify ''inline'' for any of these actions in your > /etc/shorewall/actions file and then invoke one of them in your > configuration, a fatal compilation error will be raised.In other words, if I have this: actions ~~~~~~~ C_ACTION inline action.C_ACTION ~~~~~~~~~~~~~~~ Broadcast That should bring an error, is that right? ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 11/30/12 7:23 PM, Mr Dash Four wrote:> >> If you specify ''inline'' for any of these actions in your >> /etc/shorewall/actions file and then invoke one of them in your >> configuration, a fatal compilation error will be raised. > In other words, if I have this: > > actions > ~~~~~~~ > C_ACTION inline > > action.C_ACTION > ~~~~~~~~~~~~~~~ > Broadcast > > > That should bring an error, is that right?No. If you have: actions: Broadcast inline Anywhere: Broadcast Then an error should be raised. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
> No. > > If you have: > > actions: > > Broadcast inline > > Anywhere: > > Broadcast > > Then an error should be raised.I see - in other words, trying to overwrite one of the built-actions in that list with my own and make it inline, correct? ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 11/30/2012 07:30 PM, Mr Dash Four wrote:> >> No. >> >> If you have: >> >> actions: >> >> Broadcast inline >> >> Anywhere: >> >> Broadcast >> >> Then an error should be raised. > I see - in other words, trying to overwrite one of the built-actions in that list with my own and make it inline, correct?If you have your own /etc/shorewall/action.Broadcast then it will probably be okay. If you don''t and invoke mine instead then mine will raise an error. On the other hand, if you have: actions: Drop inline Then it will work, because ''Drop'' isn''t in the forbidden list. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
> If you have your own /etc/shorewall/action.Broadcast then it will > probably be okay.Probably? If I am to test this, I need to be certain and clear on what should and should not be allowed to be included or invoked.> If you don''t and invoke mine instead then mine will > raise an error.Invoke how? ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 11/30/2012 07:43 PM, Mr Dash Four wrote:> >> If you have your own /etc/shorewall/action.Broadcast then it will >> probably be okay. > Probably? If I am to test this, I need to be certain and clear on what should and should not be allowed to be included or invoked. > >> If you don''t and invoke mine instead then mine will >> raise an error. > Invoke how?Try to use it (Broadcast in the ACTION column). That''s either directly or indirectly, and Broadcast is used by both Drop and Reject. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
>>> If you don''t and invoke mine instead then mine will >>> raise an error. >> Invoke how? > > Try to use it (Broadcast in the ACTION column). That''s either directly > or indirectly, and Broadcast is used by both Drop and Reject.Which ACTION column - in my inline user-defined action, "rules", "policy"? What should I expect to see happening if I do that (that''s not just for Broadcast, but all other built-ion actions on that list as well)? ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 11/30/2012 08:34 PM, Mr Dash Four wrote:> >>>> If you don''t and invoke mine instead then mine will raise an >>>> error. >>> Invoke how? >> >> Try to use it (Broadcast in the ACTION column). That''s either >> directly or indirectly, and Broadcast is used by both Drop and >> Reject. > Which ACTION column - in my inline user-defined action, "rules", > "policy"? What should I expect to see happening if I do that (that''s > not just for Broadcast, but all other built-ion actions on that list > as well)?You will not see anything when you in-line one of those functions in your actions file. But if you use the action in any context, or if you use any action or macro that invokes the in-lined action (either directly or indirectly), then you will see this sort of error: ... Checking /etc/shorewall/tunnels... Checking MAC Filtration -- Phase 2... Applying Policies... ERROR: The Broadcast Action may not be invoked in-line /usr/share/shorewall/action.Broadcast (line 76) from /etc/shorewall/action.Geo (line 46) In this case, my default net->all action ''Geo'' invokes Broadcast. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 12/01/2012 06:42 AM, Tom Eastep wrote:> On 11/30/2012 08:34 PM, Mr Dash Four wrote: >> >>>>> If you don''t and invoke mine instead then mine will raise an >>>>> error. >>>> Invoke how? >>> >>> Try to use it (Broadcast in the ACTION column). That''s either >>> directly or indirectly, and Broadcast is used by both Drop and >>> Reject. >> Which ACTION column - in my inline user-defined action, "rules", >> "policy"? What should I expect to see happening if I do that (that''s >> not just for Broadcast, but all other built-ion actions on that list >> as well)? > > You will not see anything when you in-line one of those functions in > your actions file. But if you use the action in any context, or if you > use any action or macro that invokes the in-lined action (either > directly or indirectly), then you will see this sort of error: > > ... > Checking /etc/shorewall/tunnels... > Checking MAC Filtration -- Phase 2... > Applying Policies... > ERROR: The Broadcast Action may not be invoked in-line > /usr/share/shorewall/action.Broadcast (line 76) > from /etc/shorewall/action.Geo (line 46) > > In this case, my default net->all action ''Geo'' invokes Broadcast.To make this a bit more deterministic, I''ve added a warning and reset the ''inline'' attribute: ... Checking /etc/shorewall/hosts... Determining Hosts in Zones... Locating Action Files... WARNING: Resetting ''inline'' option on action Broadcast -- that action may not be in-lined /usr/share/shorewall/actions.std (line 38) Checking /usr/share/shorewall/action.Drop for chain Drop... Checking /usr/share/shorewall/action.Broadcast for chain Broadcast... ... -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 12/01/2012 07:01 AM, Tom Eastep wrote:> > To make this a bit more deterministic, I''ve added a warning and reset > the ''inline'' attribute: > > ... > Checking /etc/shorewall/hosts... > Determining Hosts in Zones... > Locating Action Files... > WARNING: Resetting ''inline'' option on action Broadcast -- that > action may not be in-lined > /usr/share/shorewall/actions.std (line 38) > Checking /usr/share/shorewall/action.Drop for chain Drop... > Checking /usr/share/shorewall/action.Broadcast for chain Broadcast... > ... >Or even better: Checking /etc/shorewall/interfaces... Checking /etc/shorewall/hosts... Determining Hosts in Zones... Locating Action Files... WARNING: ''inline'' option ignored on action Broadcast -- that action may not be in-lined /etc/shorewall/actions (line 6) Checking /usr/share/shorewall/action.Drop for chain Drop... Checking /usr/share/shorewall/action.Broadcast for chain Broadcast... Checking /usr/share/shorewall/action.Invalid for chain Invalid... I restored the original order of processing the files (actions.std then actions) and added a ''noinline'' option which is specified on the 6 actions that cannot be in-lined. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
I''ve just finished testing this release and found the following issues:
1. Actions/SWITCH naming conventions
actions
~~~~~~~
C_ACTION inline
fw2net # this action is not inline!
action.fw2net
~~~~~~~~~~~~~
LOG:info ; switch:${0}_action
action.C_ACTION
~~~~~~~~~~~~~~~
LOG:info ; switch:${0}_c_action
rules
~~~~~
fw2net $FW net
results in "ERROR: Invalid switch name (%fw2net_action)" message
Further to this, if I add the following:
blrules
~~~~~~~
C_ACTION:info all all
I get "ERROR: Invalid switch name (fw2net~_c_action)"
2. action.template needs to be updated to include the SWITCH column
3. Action circular reference undetected and/or error message displayed
incomplete/wrong:
action.circ1
~~~~~~~~~~~~
$1
rules
~~~~~
circ1(circ1):info $FW net
Gets me "ERROR: Undefined parameter ($1)"
Also:
action.circ2
~~~~~~~~~~~~
circ1
rules
~~~~~
circ1(circ2):info $FW net
Gets me "ERROR: Undefined parameter ($1)"
When I have action.circ2 the same as action.circ1 as well as:
rules
~~~~~
circ1(circ2(circ1)):info $FW net
The message is "ERROR: Invalid Action (%1:info) in inline action"
(when
any of the 2 actions are inline) or "ERROR: Unknown ACTION (%1)" if
not.
All of the above messages should have given me circular reference error,
like when I reference the actions directly, i.e. something like "ERROR:
Action circ2 invoked Recursively
(circ1(circ2):info->circ2:info->circ1:info->circ2:info)" or like
the
message I am getting when both actions are set as ''inline'':
"ERROR: Macro/Inline invocations nested too deeply
/etc/shorewall/action.circ2"
4. Further to the "forbidden or not" discussion earlier, I am not sure
whether the above can be considered a bug, but, at the very least, there
seems to be inconsistency in reporting of errors/allowing inlined
actions. The following actions appear on the "forbidden" list, but are
allowed in for invocation inline: Broadcast*, Invalid, RST and TCPFlags*
In case where DropSmurfs is used inline, the error I am getting is
"ERROR: Bareword "IPv6_MULTICAST" not allowed while "strict
subs" in use
at /usr/share/shorewall/action.DropSmurfs line 80" instead of the
"ERROR: Invalid Action (XXX) in inline action"
* - This invocation is translated to a straight jump (-j <action_name>)
- inline - instead of emitting an error message.
5. Minor issue, which could be improved upon as far as optimisation of
inline actions goes:
rules
~~~~~
circ1(dropBcast) $FW net
dropBcast $FW net
generates something like:
:~comb2 [0:0]
[...]
-A fw2net -j ~comb2
-A fw2net -j ~comb2
[...]
-A ~comb2 -m addrtype --dst-type BROADCAST -j DROP
-A ~comb2 -d 224.0.0.0/4 -j DROP
Both statements for "-A fw2net ..." above should have been combined
into
a single one.
Finally, two suggestions:
1. I don''t seem to be able to invoke action with parameters *and* log
level specified as action parameter. In other words, something like:
"circ1(circ2(whatever):debug):info". It would be nice to have that
ability.
2. Having looked at "Creating an Action using an Extension Script"
(http://shorewall.net/Actions.html), it would be nice if $level and $tag
are added to the "system parameters" passed to user-defined actions
($chainref is already present there as $0). If that is possible, then
the @{XX} notation is definitely going to be needed.
------------------------------------------------------------------------------
Keep yourself connected to Go Parallel:
INSIGHTS What''s next for parallel hardware, programming and related
areas?
Interviews and blogs by thought leaders keep you ahead of the curve.
http://goparallel.sourceforge.net
> 2. Having looked at "Creating an Action using an Extension Script" > (http://shorewall.net/Actions.html), it would be nice if $level and > $tag are added to the "system parameters" passed to user-defined > actions ($chainref is already present there as $0). If that is > possible, then the @{XX} notation is definitely going to be needed.Forgot to add: Sometimes, when I use "$0" in my actions I am getting "invalid parameter ($0)" error, but when I use "${0}" instead, all is well - is the use of curly braces enforced by shorewall? ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 12/1/12 9:32 AM, Mr Dash Four wrote:> I''ve just finished testing this release and found the following issues: > > 1. Actions/SWITCH naming conventions > > actions > ~~~~~~~ > C_ACTION inline > fw2net # this action is not inline! > > action.fw2net > ~~~~~~~~~~~~~ > LOG:info ; switch:${0}_action > > action.C_ACTION > ~~~~~~~~~~~~~~~ > LOG:info ; switch:${0}_c_actionYou are using ${0} -- if you use @{0} then it works (but you will need to wait until RC 1; in Beta3, just use @). Within a SWITCH, @{0} (@ in Beta 3) expands to the name of the current chain with non-alphanumeric characters except ''_'' and ''-'' suppressed. ${0} always expands to the full name of the chain.> > rules > ~~~~~ > fw2net $FW net > > results in "ERROR: Invalid switch name (%fw2net_action)" message > > Further to this, if I add the following: > > blrules > ~~~~~~~ > C_ACTION:info all all > > I get "ERROR: Invalid switch name (fw2net~_c_action)"Same issue.> > 2. action.template needs to be updated to include the SWITCH columnUpdated; thanks> > 3. Action circular reference undetected and/or error message displayed > incomplete/wrong: > > action.circ1 > ~~~~~~~~~~~~ > $1 > > rules > ~~~~~ > circ1(circ1):info $FW net > > Gets me "ERROR: Undefined parameter ($1)"You are not getting far enough for the recursion to be discovered. If you change: action.circ1 ^^^^^^^^^^^^ $1($1) Then you get ERROR: Action circ1 invoked Recursively (circ1(circ1):info->circ1(circ1):info) The way you have coded the action, you are invoking circ1 with no parameters; as a result, expansion of $1 fails in the pre-processor.> > Also: > > action.circ2 > ~~~~~~~~~~~~ > circ1 > > rules > ~~~~~ > circ1(circ2):info $FW net > > Gets me "ERROR: Undefined parameter ($1)"Again, you are invoking circ1 without an argument.> > When I have action.circ2 the same as action.circ1 as well as: > > rules > ~~~~~ > circ1(circ2(circ1)):info $FW net > > The message is "ERROR: Invalid Action (%1:info) in inline action" (when > any of the 2 actions are inline) or "ERROR: Unknown ACTION (%1)" if not. > > All of the above messages should have given me circular reference error, > like when I reference the actions directly, i.e. something like "ERROR: > Action circ2 invoked Recursively > (circ1(circ2):info->circ2:info->circ1:info->circ2:info)" or like the > message I am getting when both actions are set as ''inline'': > "ERROR: Macro/Inline invocations nested too deeply > /etc/shorewall/action.circ2"Again, if the body of the two actions is $1($1) then the error message is: ERROR: Action circ1 invoked Recursively (circ1(circ1):info->circ1(circ1):info) /home/teastep/shorewall /regressionLibrary/4.5.10/inline2/action.circ1 (line 1) from /home/teastep/shorewall/regressionLibrary/4.5.10/inline2 /rules (line 46) Given that variable expansion is done in the pre-processor which has no knowledge of the context, there isn''t much I can do about this.> > 4. Further to the "forbidden or not" discussion earlier, I am not sure > whether the above can be considered a bug, but, at the very least, there > seems to be inconsistency in reporting of errors/allowing inlined > actions. The following actions appear on the "forbidden" list, but are > allowed in for invocation inline: Broadcast*, Invalid, RST and TCPFlags* > > In case where DropSmurfs is used inline, the error I am getting is > "ERROR: Bareword "IPv6_MULTICAST" not allowed while "strict subs" in use > at /usr/share/shorewall/action.DropSmurfs line 80" instead of the > "ERROR: Invalid Action (XXX) in inline action" > > * - This invocation is translated to a straight jump (-j <action_name>) > - inline - instead of emitting an error message.This is a non-issue now that ''inline'' is ignored for these actions.> > 5. Minor issue, which could be improved upon as far as optimisation of > inline actions goes: > > rules > ~~~~~ > circ1(dropBcast) $FW net > dropBcast $FW net > > generates something like: > > :~comb2 [0:0] > [...] > -A fw2net -j ~comb2 > -A fw2net -j ~comb2 > [...] > -A ~comb2 -m addrtype --dst-type BROADCAST -j DROP > -A ~comb2 -d 224.0.0.0/4 -j DROP > > Both statements for "-A fw2net ..." above should have been combined into > a single one.Doesn''t setting OPTIMIZE=31 remove the duplicate rule?> > Finally, two suggestions: > > 1. I don''t seem to be able to invoke action with parameters *and* log > level specified as action parameter. In other words, something like: > "circ1(circ2(whatever):debug):info". It would be nice to have that ability. > > 2. Having looked at "Creating an Action using an Extension Script" > (http://shorewall.net/Actions.html), it would be nice if $level and $tag > are added to the "system parameters" passed to user-defined actions > ($chainref is already present there as $0). If that is possible, then > the @{XX} notation is definitely going to be needed. >I''ll add these to the list of things to look at for 4.5.11. Regards, -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 12/1/12 11:09 AM, Tom Eastep wrote:> On 12/1/12 9:32 AM, Mr Dash Four wrote: >> I''ve just finished testing this release and found the following issues: >> >> 1. Actions/SWITCH naming conventions >> >> actions >> ~~~~~~~ >> C_ACTION inline >> fw2net # this action is not inline! >> >> action.fw2net >> ~~~~~~~~~~~~~ >> LOG:info ; switch:${0}_action >> >> action.C_ACTION >> ~~~~~~~~~~~~~~~ >> LOG:info ; switch:${0}_c_action > > You are using ${0} -- if you use @{0} then it works (but you will need > to wait until RC 1; in Beta3, just use @). > > Within a SWITCH, @{0} (@ in Beta 3) expands to the name of the current > chain with non-alphanumeric characters except ''_'' and ''-'' suppressed. > ${0} always expands to the full name of the chain. > >> >> rules >> ~~~~~ >> fw2net $FW net >> >> results in "ERROR: Invalid switch name (%fw2net_action)" message >> >> Further to this, if I add the following: >> >> blrules >> ~~~~~~~ >> C_ACTION:info all all >> >> I get "ERROR: Invalid switch name (fw2net~_c_action)" > > Same issue. > >> >> 2. action.template needs to be updated to include the SWITCH column > > Updated; thanks > >> >> 3. Action circular reference undetected and/or error message displayed >> incomplete/wrong: >> >> action.circ1 >> ~~~~~~~~~~~~ >> $1 >> >> rules >> ~~~~~ >> circ1(circ1):info $FW net >> >> Gets me "ERROR: Undefined parameter ($1)" > > You are not getting far enough for the recursion to be discovered. If > you change: > > action.circ1 > ^^^^^^^^^^^^ > $1($1) > > Then you get > > ERROR: Action circ1 invoked Recursively > (circ1(circ1):info->circ1(circ1):info) > > The way you have coded the action, you are invoking circ1 with no > parameters; as a result, expansion of $1 fails in the pre-processor. > >>I took another look at this and I''m able to do something about it. Patch attached. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 12/1/12 9:41 AM, Mr Dash Four wrote:> >> 2. Having looked at "Creating an Action using an Extension Script" >> (http://shorewall.net/Actions.html), it would be nice if $level and >> $tag are added to the "system parameters" passed to user-defined >> actions ($chainref is already present there as $0). If that is >> possible, then the @{XX} notation is definitely going to be needed. > Forgot to add: Sometimes, when I use "$0" in my actions I am getting > "invalid parameter ($0)" error, but when I use "${0}" instead, all is > well - is the use of curly braces enforced by shorewall?No. If you can come up with a simple test case, I''ll take a look. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 12/1/12 12:28 PM, Tom Eastep wrote:> On 12/1/12 9:41 AM, Mr Dash Four wrote: >> >>> 2. Having looked at "Creating an Action using an Extension Script" >>> (http://shorewall.net/Actions.html), it would be nice if $level and >>> $tag are added to the "system parameters" passed to user-defined >>> actions ($chainref is already present there as $0). If that is >>> possible, then the @{XX} notation is definitely going to be needed. >> Forgot to add: Sometimes, when I use "$0" in my actions I am getting >> "invalid parameter ($0)" error, but when I use "${0}" instead, all is >> well - is the use of curly braces enforced by shorewall? > > No. If you can come up with a simple test case, I''ll take a look. >If $0 is followed by ''_'' (e.g., $0_switch) then you must use ${0}. The preprocessor is looking for ''$'' followed by Perl ''word'' characters (alphanumerics + ''_''). -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 12/1/12 12:33 PM, Tom Eastep wrote:> On 12/1/12 12:28 PM, Tom Eastep wrote: >> On 12/1/12 9:41 AM, Mr Dash Four wrote: >>> >>>> 2. Having looked at "Creating an Action using an Extension Script" >>>> (http://shorewall.net/Actions.html), it would be nice if $level and >>>> $tag are added to the "system parameters" passed to user-defined >>>> actions ($chainref is already present there as $0). If that is >>>> possible, then the @{XX} notation is definitely going to be needed. >>> Forgot to add: Sometimes, when I use "$0" in my actions I am getting >>> "invalid parameter ($0)" error, but when I use "${0}" instead, all is >>> well - is the use of curly braces enforced by shorewall? >> >> No. If you can come up with a simple test case, I''ll take a look. >> > > If $0 is followed by ''_'' (e.g., $0_switch) then you must use ${0}. The > preprocessor is looking for ''$'' followed by Perl ''word'' characters > (alphanumerics + ''_'').Here''s a patch that avoids this issue. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
On 12/1/12 11:09 AM, Tom Eastep wrote:> On 12/1/12 9:32 AM, Mr Dash Four wrote:>> Finally, two suggestions: >> >> 1. I don''t seem to be able to invoke action with parameters *and* log >> level specified as action parameter. In other words, something like: >> "circ1(circ2(whatever):debug):info". It would be nice to have that ability.The attached patch seems to handle this case correctly. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: INSIGHTS What''s next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net
>> If $0 is followed by ''_'' (e.g., $0_switch) then you must use ${0}. The >> preprocessor is looking for ''$'' followed by Perl ''word'' characters >> (alphanumerics + ''_''). > > > Here''s a patch that avoids this issue.That was indeed the case - perl seems to handle $X variables a bit differently from bash, which was a bit unexpected for me. The patch you attached does indeed fix this issue. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
>>> 1. I don''t seem to be able to invoke action with parameters *and* log >>> level specified as action parameter. In other words, something like: >>> "circ1(circ2(whatever):debug):info". It would be nice to have that ability. > > The attached patch seems to handle this case correctly.Yep, it does the job, though I spotted another issue: when I issue a LOG statement (as part of action - inline or otherwise) the generated LOG iptables statement does not contain all the "--log-*" options I have used in, say, my default policy for example. Is there a way to control these options (that will also be needed when you make available $tag later on)? ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
> You are using ${0} -- if you use @{0} then it works (but you will need > to wait until RC 1; in Beta3, just use @). > > Within a SWITCH, @{0} (@ in Beta 3) expands to the name of the current > chain with non-alphanumeric characters except ''_'' and ''-'' suppressed. > ${0} always expands to the full name of the chain.Is this behaviour explained/documented anywhere, particularly the distinction between ''@'' and ''$0''? Substituting $0 with @ does indeed work.>> 4. Further to the "forbidden or not" discussion earlier, I am not sure >> whether the above can be considered a bug, but, at the very least, there >> seems to be inconsistency in reporting of errors/allowing inlined >> actions. The following actions appear on the "forbidden" list, but are >> allowed in for invocation inline: Broadcast*, Invalid, RST and TCPFlags* >> >> In case where DropSmurfs is used inline, the error I am getting is >> "ERROR: Bareword "IPv6_MULTICAST" not allowed while "strict subs" in use >> at /usr/share/shorewall/action.DropSmurfs line 80" instead of the >> "ERROR: Invalid Action (XXX) in inline action" >> >> * - This invocation is translated to a straight jump (-j <action_name>) >> - inline - instead of emitting an error message. > > This is a non-issue now that ''inline'' is ignored for these actions.Please elaborate.> >> 5. Minor issue, which could be improved upon as far as optimisation of >> inline actions goes: >> >> rules >> ~~~~~ >> circ1(dropBcast) $FW net >> dropBcast $FW net >> >> generates something like: >> >> :~comb2 [0:0] >> [...] >> -A fw2net -j ~comb2 >> -A fw2net -j ~comb2 >> [...] >> -A ~comb2 -m addrtype --dst-type BROADCAST -j DROP >> -A ~comb2 -d 224.0.0.0/4 -j DROP >> >> Both statements for "-A fw2net ..." above should have been combined into >> a single one. > > Doesn''t setting OPTIMIZE=31 remove the duplicate rule?No. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
> I took another look at this and I''m able to do something about it. > > Patch attached.Yep, that does the trick. One clarification though: my understanding was that action parameters are separated by commas. If so, when I have action circ1 consisting of just "$1" and execute "circ1(circ2(whatever):debug):info" then $1 becomes "circ2(whatever):debug", isn''t that so? If that is the case, the correct statement is $1, not as you suggested "$1($1)" in your previous post. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
On 12/1/12 4:53 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:> >> I took another look at this and I''m able to do something about it. >> >> Patch attached. >Yep, that does the trick. > >One clarification though: my understanding was that action parameters are >separated by commas. If so, when I have action circ1 consisting of just >"$1" and execute "circ1(circ2(whatever):debug):info" then $1 becomes >"circ2(whatever):debug", isn''t that so? If that is the case, the correct >statement is $1, not as you suggested "$1($1)" in your previous post.In your example, both circa and circa were simply $1; eventually, you run out of parameters and end trying to expand a rule that is just ''circa2''; in that invocation, there is no $1. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
On 12/1/12 4:52 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:> >>> If $0 is followed by ''_'' (e.g., $0_switch) then you must use ${0}. The >>> preprocessor is looking for ''$'' followed by Perl ''word'' characters >>> (alphanumerics + ''_''). >> >> >> Here''s a patch that avoids this issue. >That was indeed the case - perl seems to handle $X variables a bit >differently from bash, which was a bit unexpected for me. The patch you >attached does indeed fix this issue.The two handle (scalar) variables the same, at least on the right side of assignments. My regular expression was just looking for any sequence of ''word'' characters after ''$''; it wasn''t distinguishing the case where the charater after ''$'' was a digit from the other cases. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
On 12/1/12 4:53 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:>> You are using ${0} -- if you use @{0} then it works (but you will need >> to wait until RC 1; in Beta3, just use @). >> >> Within a SWITCH, @{0} (@ in Beta 3) expands to the name of the current >> chain with non-alphanumeric characters except ''_'' and ''-'' suppressed. >> ${0} always expands to the full name of the chain. >Is this behaviour explained/documented anywhere, particularly the >distinction between ''@'' and ''$0''? Substituting $0 with @ does indeed work.Not in Beta 3; remember that ''@'' isn''t documented in Beta 3.> >>> 4. Further to the "forbidden or not" discussion earlier, I am not sure >>> whether the above can be considered a bug, but, at the very least, >>>there >>> seems to be inconsistency in reporting of errors/allowing inlined >>> actions. The following actions appear on the "forbidden" list, but are >>> allowed in for invocation inline: Broadcast*, Invalid, RST and >>>TCPFlags* >>> >>> In case where DropSmurfs is used inline, the error I am getting is >>> "ERROR: Bareword "IPv6_MULTICAST" not allowed while "strict subs" in >>>use >>> at /usr/share/shorewall/action.DropSmurfs line 80" instead of the >>> "ERROR: Invalid Action (XXX) in inline action" >>> >>> * - This invocation is translated to a straight jump (-j >>><action_name>) >>> - inline - instead of emitting an error message. >> >> This is a non-issue now that ''inline'' is ignored for these actions. >Please elaborate.My post of this afternoon explained that in RC 1, the compile will ignore (with a warning), ''inline'' on one of these actions. So I don''t believe that there is any inconsistency once that release is available.> >> >>> 5. Minor issue, which could be improved upon as far as optimisation of >>> inline actions goes: >>> >>> rules >>> ~~~~~ >>> circ1(dropBcast) $FW net >>> dropBcast $FW net >>> >>> generates something like: >>> >>> :~comb2 [0:0] >>> [...] >>> -A fw2net -j ~comb2 >>> -A fw2net -j ~comb2 >>> [...] >>> -A ~comb2 -m addrtype --dst-type BROADCAST -j DROP >>> -A ~comb2 -d 224.0.0.0/4 -j DROP >>> >>> Both statements for "-A fw2net ..." above should have been combined >>>into >>> a single one. >> >> Doesn''t setting OPTIMIZE=31 remove the duplicate rule? >No.Okay -- I''ll look at that tomorrow. Thanks for testing, -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
On 12/01/2012 05:29 PM, Tom Eastep wrote:>>>> 4. Further to the "forbidden or not" discussion earlier, I am not sure >>>> whether the above can be considered a bug, but, at the very least, >>>> there >>>> seems to be inconsistency in reporting of errors/allowing inlined >>>> actions. The following actions appear on the "forbidden" list, but are >>>> allowed in for invocation inline: Broadcast*, Invalid, RST and >>>> TCPFlags* >>>> >>>> In case where DropSmurfs is used inline, the error I am getting is >>>> "ERROR: Bareword "IPv6_MULTICAST" not allowed while "strict subs" in >>>> use >>>> at /usr/share/shorewall/action.DropSmurfs line 80" instead of the >>>> "ERROR: Invalid Action (XXX) in inline action" >>>> >>>> * - This invocation is translated to a straight jump (-j >>>> <action_name>) >>>> - inline - instead of emitting an error message. >>> >>> This is a non-issue now that ''inline'' is ignored for these actions. >> Please elaborate. > > My post of this afternoon explained that in RC 1, the compile will ignore > (with a warning), ''inline'' on one of these actions. So I don''t believe > that there is any inconsistency once that release is available. >Ah -- but there was a bug in action.DropSmurfs; patch attached. Thanks, -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
On 12/01/2012 04:53 PM, Mr Dash Four wrote:> >>>> 1. I don''t seem to be able to invoke action with parameters >>>> *and* log level specified as action parameter. In other words, >>>> something like: "circ1(circ2(whatever):debug):info". It would >>>> be nice to have that ability. >> >> The attached patch seems to handle this case correctly. > Yep, it does the job, though I spotted another issue: when I issue a > LOG statement (as part of action - inline or otherwise) the generated > LOG iptables statement does not contain all the "--log-*" options I > have used in, say, my default policy for example. Is there a way to > control these options (that will also be needed when you make > available $tag later on)?When LOG (NFLOG,ULOG) is used anywhere, the two things that appear in the --log-prefix are: - The Chain Name - The ''disposition'' of the rule that generated the message If LOGTAGONLY=Yes is specified and a tag is supplied along with the log level, then the tag replaces the Chain Name. I do that to break out several countries for logging within my own net->all default action: ?if $GEOIP_MATCH DROP:$LOG:China ^CN DROP:$LOG:Russia ^RU DROP:$LOG:USA ^US ... ?endif That produces messages such as: Dec 2 06:39:40 USA DROP IN=eth1 OUT=br0 SRC=76.121.47.47 DST=70.90.191.124 LEN=52 TOS=00 PREC=0x20 TTL=121 ID=28849 DF PROTO=TCP SPT=53499 DPT=37507 SEQ=165394300 ACK=0 WINDOW=8192 SYN URGP=0 MARK=0 Dec 2 06:39:42 USA DROP IN=eth1 OUT=br0 SRC=76.121.47.47 DST=70.90.191.124 LEN=61 TOS=00 PREC=0x20 TTL=121 ID=28950 PROTO=UDP SPT=10914 DPT=37507 LEN=41 MARK=0 Dec 2 06:44:14 China DROP IN=eth1 OUT=br0 SRC=58.209.94.102 DST=70.90.191.124 LEN=60 TOS=00 PREC=0x20 TTL=47 ID=58478 DF PROTO=TCP SPT=54540 DPT=23 SEQ=4222062522 ACK=0 WINDOW=5440 SYN URGP=0 MARK=0 Dec 2 06:44:15 USA DROP IN=eth1 OUT=br0 SRC=76.121.47.47 DST=70.90.191.124 LEN=52 TOS=00 PREC=0x20 TTL=121 ID=14961 DF PROTO=TCP SPT=54286 DPT=37507 SEQ=408863359 ACK=0 WINDOW=8192 SYN URGP=0 MARK=0 Dec 2 06:44:15 USA DROP IN=eth1 OUT=br0 SRC=76.121.47.47 DST=70.90.191.124 LEN=61 TOS=00 PREC=0x20 TTL=121 ID=14960 PROTO=UDP SPT=10914 DPT=37507 LEN=41 MARK=0 If LOGTAGONLY=No and a tag is supplied, the tag follows the disposition. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
On 12/01/2012 04:53 PM, Mr Dash Four wrote:>> >>> 5. Minor issue, which could be improved upon as far as optimisation of >>> inline actions goes: >>> >>> rules >>> ~~~~~ >>> circ1(dropBcast) $FW net >>> dropBcast $FW net >>> >>> generates something like: >>> >>> :~comb2 [0:0] >>> [...] >>> -A fw2net -j ~comb2 >>> -A fw2net -j ~comb2 >>> [...] >>> -A ~comb2 -m addrtype --dst-type BROADCAST -j DROP >>> -A ~comb2 -d 224.0.0.0/4 -j DROP >>> >>> Both statements for "-A fw2net ..." above should have been combined into >>> a single one. >> >> Doesn''t setting OPTIMIZE=31 remove the duplicate rule? > No. >I have: actions: -------- run action.run: ----------- $1 rules: ------ ... run(dropBcast) $FW net dropBcast $FW net shorewall.conf: --------------- ... OPTIMIZE=31 ... That generates the following rules in fw2net: -A fw2net -p udp --dport 67:68 -j ACCEPT -A fw2net -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT -A fw2net -p 17 --dport 53 -j ACCEPT -m comment --comment "DNS" -A fw2net -p 6 --dport 53 -j ACCEPT -m comment --comment "DNS" -A fw2net -p 1 -j ACCEPT -A fw2net -j run -A fw2net -m addrtype --dst-type BROADCAST -j DROP -A fw2net -d 224.0.0.0/4 -j DROP -A fw2net -j Reject -A fw2net -j LOG --log-level 6 --log-prefix "Shorewall:fw2net:REJECT:" -A fw2net -g reject If I replace the rules with: run(Broadcast(DROP)) $FW net Broadcast(DROP) $FW net Then I get: -A fw2net -p udp --dport 67:68 -j ACCEPT -A fw2net -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT -A fw2net -p 17 --dport 53 -j ACCEPT -m comment --comment "DNS" -A fw2net -p 6 --dport 53 -j ACCEPT -m comment --comment "DNS" -A fw2net -p 1 -j ACCEPT -A fw2net -j run -A fw2net -j ~comb0 -A fw2net -j Reject -A fw2net -j LOG --log-level 6 --log-prefix "Shorewall:fw2net:REJECT:" -A fw2net -g reject ... -A run -j ~comb0 ... -A ~comb0 -m addrtype --dst-type BROADCAST -j DROP -A ~comb0 -m addrtype --dst-type MULTICAST -j DROP -A ~comb0 -m addrtype --dst-type ANYCAST -j DROP -A ~comb0 -d 224.0.0.0/4 -j DROP If I inline the run action, I get: -A fw2net -j ~comb0 -m comment --comment "run" -A fw2net -j ~comb0 If I turn off AUTOCOMMENT (after fixing AUTOCOMMENT=No with the attached patch), I get: -A fw2net -p udp --dport 67:68 -j ACCEPT -A fw2net -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT -A fw2net -p 17 --dport 53 -j ACCEPT -A fw2net -p 6 --dport 53 -j ACCEPT -A fw2net -p 1 -j ACCEPT -A fw2net -j ~comb0 -A fw2net -j ~comb0 -A fw2net -j Reject -A fw2net -j LOG --log-level 6 --log-prefix "Shorewall:fw2net:REJECT:" -A fw2net -g reject (Finally!) When optimize level 16 is requested, the optimizer currently deletes duplicate rules only in the ''raw'' table. There is quite a bit of work and lots of testing to be done to delete them in the other table chains, because there are cases in those chains where duplicate rules must not deleted. As a trivial example: -A chain -j CONNMARK --restore-mark -A chain -m mark ! --mark 0 RETURN -A chain -s 1.2.3.4 -j MARK --set-mark 1 -A chain -s 2.3.4.5 -j MARK --set-mark 2 -A chain -m mark ! --mark 0 RETURN -A chain -j MARK --set-mark 3 -A chain -j CONNMARK --save-mark It''s clearly a bad idea to remove the duplicate RETURN rule. I''ll think about working on this for 4.5.11. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
On 12/02/2012 08:57 AM, Tom Eastep wrote:> When optimize level 16 is requested, the optimizer currently deletes > duplicate rules only in the ''raw'' table. There is quite a bit of work > and lots of testing to be done to delete them in the other table chains, > because there are cases in those chains where duplicate rules must not > deleted. As a trivial example: > > -A chain -j CONNMARK --restore-mark > -A chain -m mark ! --mark 0 RETURN > -A chain -s 1.2.3.4 -j MARK --set-mark 1 > -A chain -s 2.3.4.5 -j MARK --set-mark 2 > -A chain -m mark ! --mark 0 RETURN > -A chain -j MARK --set-mark 3 > -A chain -j CONNMARK --save-mark > > It''s clearly a bad idea to remove the duplicate RETURN rule. > > I''ll think about working on this for 4.5.11. >In the meantime, here''s a very conservative patch that deletes adjacent duplicates in the other tables. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
> When LOG (NFLOG,ULOG) is used anywhere, the two things that appear in > the --log-prefix are: > > - The Chain Name > - The ''disposition'' of the rule that generated the message >After a bit of experimenting, I just realised that these options are available as part of the log level. In other words, I can specify "LOG:info(uid,tcp_options,ip_options,macdecode,tcp_sequence) $FW net" to get these included. Up until now, I was under the impression that the options in brackets are only available as part of the log level specified in the policy file, but that seems not to be the case, so I am happy. Is this documented anywhere (I am aware that it is for "policy")? ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
On 12/2/12 3:51 PM, Mr Dash Four wrote:> >> When LOG (NFLOG,ULOG) is used anywhere, the two things that appear >> in the --log-prefix are: >> >> - The Chain Name - The ''disposition'' of the rule that generated the >> message >> > After a bit of experimenting, I just realised that these options are > available as part of the log level. In other words, I can specify > "LOG:info(uid,tcp_options,ip_options,macdecode,tcp_sequence) $FW net" > to get these included. Up until now, I was under the impression that > the options in brackets are only available as part of the log level > specified in the policy file, but that seems not to be the case, so I > am happy. Is this documented anywhere (I am aware that it is for > "policy")?See http://www.shorewall.net/shorewall_logging.html -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/
> See http://www.shorewall.net/shorewall_logging.htmlThanks, pretty much what I needed. ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net