Beta 4 is now available for testing.]
Problems Corrected since Beta 3:
1) If a chain consisted of a single RETURN rule, optimize level 4
would handle it incorrectly by moving the RETURN rule to the
chain(s) that jumped to the single-rule chain. The optimizer now
simply eliminate the chain and rule.
New in Beta 4:
As part of this change, the optimizer now deletes trailing RETURN
rules from chains.
New Features since Beta 3:
1) There are now ''Related'', ''Untracked'' and
''Established'' actions that
match packets in the RELATED and UNTRACKED states respectively.
These actions are in-line and have a single parameter that
specifies the action to be taken. The action may be anything that
is valid in the ACTION column of the rules file.
As part of this change, action.Invalid, action.NotSyn and
action.RST are also inline and can accept an arbitrary action as an
argument. The ''audit'' parameter, while still accepted, is
deprecated in favor of passing ''A_ACCEPT'' etc. directly to
the
inline.
2) The preceding enhancement required infrastructure for allowing
BEGIN PERL...END PERL to function in the body of an inline action.
use Shorewall::Rules;
perl_action_helper( $target, $matches )
$target is the target of the rule and may include log level and
tag (e.g, ''DROP:info:foo'').
$matches is a string containing one or more ip[6]tables
matches. The string must end with a space.
Example: "-m conntrack --state ESTABLISHED ".
The function returns true.
This function may be called in both inline and regular actions. In
an inline action, the matches from the invoking rule (SOURCE, DEST,
etc) are applied (in addition to the match(s) passed). In a regular
action only the passed matches are applied to the rule.
3) To allow finer-grained selection of the connection-tracking states
that are passed through blacklists (both dynamic and static), a
BLACKLIST option has been added in shorewall.conf and
shorewall6.conf.
The BLACKLISTNEWONLY option is now deprecated. A ''shorewall
update''
( ''shorewall6 update'' ) will replace the BLACKLISTNEWONLY
option
with the equivalent BLACKLIST option.
Thank you for testing,
-Tom
--
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_jan
On 01/31/2013 04:09 PM, Tom Eastep wrote:> New Features since Beta 3: > > 1) There are now ''Related'', ''Untracked'' and ''Established'' actions that > match packets in the RELATED and UNTRACKED states respectively. >There is a defect in /usr/share/shorewall/action.Established. This line: "$globals{STATEMATCH} INVALID ", # Matches should read: "$globals{STATEMATCH} ESTABLISHED ", # Matches -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
> 1) If a chain consisted of a single RETURN rule, optimize level 4 > would handle it incorrectly by moving the RETURN rule to the > chain(s) that jumped to the single-rule chain. The optimizer now > simply eliminate the chain and rule.That works now.> As part of this change, the optimizer now deletes trailing RETURN > rules from chains.I wasn''t able to test this. How is this produced?> 1) There are now ''Related'', ''Untracked'' and ''Established'' actions that > match packets in the RELATED and UNTRACKED states respectively. > > These actions are in-line and have a single parameter that > specifies the action to be taken. The action may be anything that > is valid in the ACTION column of the rules file.1. Issue warning/error when state rules will never execute: rules ~~~~~ SECTION ESTABLISHED Invalid(DROP) all all or shorewall.conf ~~~~~~~~~~~~~~ BLACKLIST="NEW,UNTRACKED" blrules ~~~~~~~ Invalid(DROP) all all The above Invalid(DROP) rules will never execute since the outline state is NEW (rules) and "NEW,UNTRACKED" (blrules). 2. The old dropInvalid and the new Invalid/Established/... actions are not optimised: rules ~~~~~ SECTION ALL Invalid(DROP) fw net [...] SECTION NEW dropInvalid fw net creates the following: -A fw2net -m conntrack --ctstate INVALID -j DROP -A fw2net -m conntrack --ctstate ESTABLISHED -j ACCEPT -A fw2net -m conntrack --ctstate RELATED -j +fw2net -A fw2net -m conntrack --ctstate INVALID -j DROP Please note that there is no SECTION INVALID present in my "rules" file. The case is also the same when I have something like: rules ~~~~~ SECTION ALL Invalid(DROP) all all Invalid(A_DROP) fw net [...] SECTION INVALID DROP fw net udp ACCEPT all all [...] SECTION NEW dropInvalid all all [...] This produces: [...] -A _fw2net -p 17 -j DROP -A _fw2net -j ACCEPT [...] -A fw2net -m conntrack --ctstate INVALID -j DROP -A fw2net -m conntrack --ctstate INVALID -j A_DROP -A fw2net -m conntrack --ctstate ESTABLISHED -j ACCEPT -A fw2net -m conntrack --ctstate RELATED -j ACCEPT -A fw2net -m conntrack --ctstate INVALID -j _fw2net -A fw2net -m conntrack --ctstate UNTRACKED -j DROP -A fw2net -m conntrack --ctstate INVALID -j DROP All of the above "INVALID" state statements should have been optimised or combined in _fw2net. 3. Internal error: ERROR: "process_rule1" is not exported by the Shorewall::Rules module rules ~~~~~ SECTION RELATED Related(IELOG(-,fw2NeT,2)) $FW net IELOG is inline action taking 3 parameters: (custom) disposition, (custom) chain and NFLOG class number. The full error message is: ERROR: "process_rule1" is not exported by the Shorewall::Rules module Can''t continue after import errors at /usr/share/shorewall/action.Related line 40 BEGIN failed--compilation aborted at /usr/share/shorewall/action.Related line 40. I get the same error with "Established" and "Untracked". "Invalid" passes, though I have another 2 problems: -A +fw2net -m conntrack --ctstate INVALID -j LOG --log-tcp-options --log-ip-options --log-macdecode --log-tcp-sequence --log-uid --log-level 6 --log-prefix "Shorewall:fw2NeT::" -A +fw2net -m conntrack --ctstate INVALID -j NFLOG --nflog-group 2 --nflog-range 0 --nflog-threshold 1 --nflog-prefix "Shorewall:fw2NeT::" Notice the "--cstate INVALID" matches - these could all be optimised in a single chain. I have tried custom actions with more statements (7 to be exact) and I had the same number of "--cstate INVALID" matches, which is wrong. I am ignoring the fact that in the RELATED state, the above statements will never execute (as the match is against INVALID), but this has already been raised. so I am not going to repeat it again. The second problem is this: rules ~~~~~ SECTION RELATED IELOG(-,fw2NeT,2) Invalid(IELOG(-,fw2NeT,2)) $FW net produces: -A +fw2net -m conntrack -j LOG --log-tcp-options --log-ip-options --log-macdecode --log-tcp-sequence --log-uid --log-level 6 --log-prefix "Shorewall:fw2NeT::" -A +fw2net -m conntrack -j NFLOG --nflog-group 2 --nflog-range 0 --nflog-threshold 1 --nflog-prefix "Shorewall:fw2NeT::" In other words, not even a hint of "--cstate INVALID" (it could have been "optimised away" by mistake). In general, if the action above Invalid seems to be inline, the whole "Invalid(...)" statement seems to be totally ignored. 4. "normal" (not inline) actions now seem to be ignored by shorewall: actions ~~~~~~~ IELOG inline ELOG Please note that I have action.ELOG symlinked to action.IELOG. In other words, these are exactly the same, only the definition in "actions" is different. rules ~~~~~ SECTION NEW ELOG(-,fw2NeT,2) $FW net [...] produces: -A fw2net -m conntrack --ctstate ESTABLISHED -j ACCEPT -A fw2net -m conntrack --ctstate RELATED -j +fw2net -A fw2net [...] Note the last statement above - that won''t even compile! The same nonsensical statement is produced when I have the above ELOG statement placed in SECTION ALL. If I place ELOG in RELATED, that is completely ignored (as if it isn''t there). Same goes for: - SECTION ESTABLISHED (the rule produced is "-A fw2net -m conntrack --ctstate ESTABLISHED"); - SECTION UNTRACKED (the rule produced is "-A fw2net -m conntrack --ctstate UNTRACKED -j ~comb0" where "~comb0" consists of a single "-j DROP" statement); and - SECTION INVALID (the rule produced is similar: "-A fw2net -m conntrack --ctstate INVALID -j ~comb0").> 3) To allow finer-grained selection of the connection-tracking states > that are passed through blacklists (both dynamic and static), a > BLACKLIST option has been added in shorewall.conf and > shorewall6.conf.That works to perfection now. ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
On 02/01/2013 08:01 PM, Mr Dash Four wrote:> >> As part of this change, the optimizer now deletes trailing RETURN >> rules from chains.SECTION INVALID DROP ... ACCEPT ... RETURN foo bar> I wasn''t able to test this. How is this produced? > >> 1) There are now ''Related'', ''Untracked'' and ''Established'' actions that >> match packets in the RELATED and UNTRACKED states respectively. >> >> These actions are in-line and have a single parameter that >> specifies the action to be taken. The action may be anything that >> is valid in the ACTION column of the rules file. > 1. Issue warning/error when state rules will never execute: > > rules > ~~~~~ > SECTION ESTABLISHED > Invalid(DROP) all all > > or > > shorewall.conf > ~~~~~~~~~~~~~~ > BLACKLIST="NEW,UNTRACKED" > > blrules > ~~~~~~~ > Invalid(DROP) all all > > The above Invalid(DROP) rules will never execute since the outline state is NEW (rules) and "NEW,UNTRACKED" (blrules).Yes -- I admit that the compiler doesn''t try to understand unmatchable rules; this is only one example of that lack.> > 2. The old dropInvalid and the new Invalid/Established/... actions are not optimised: > > rules > ~~~~~ > SECTION ALL > Invalid(DROP) fw net > [...] > SECTION NEW > dropInvalid fw net > > creates the following: > > -A fw2net -m conntrack --ctstate INVALID -j DROP > -A fw2net -m conntrack --ctstate ESTABLISHED -j ACCEPT > -A fw2net -m conntrack --ctstate RELATED -j +fw2net > -A fw2net -m conntrack --ctstate INVALID -j DROP > > Please note that there is no SECTION INVALID present in my "rules" file. The case is also the same when I have something like: > > rules > ~~~~~ > SECTION ALL > Invalid(DROP) all all > Invalid(A_DROP) fw net > [...] > SECTION INVALID > DROP fw net udp > ACCEPT all all > [...] > SECTION NEW > dropInvalid all all > [...] > > This produces: > > [...] > -A _fw2net -p 17 -j DROP > -A _fw2net -j ACCEPT > [...] > -A fw2net -m conntrack --ctstate INVALID -j DROP > -A fw2net -m conntrack --ctstate INVALID -j A_DROP > -A fw2net -m conntrack --ctstate ESTABLISHED -j ACCEPT > -A fw2net -m conntrack --ctstate RELATED -j ACCEPT > -A fw2net -m conntrack --ctstate INVALID -j _fw2net > -A fw2net -m conntrack --ctstate UNTRACKED -j DROP > -A fw2net -m conntrack --ctstate INVALID -j DROP > > All of the above "INVALID" state statements should have been optimised or combined in _fw2net.I''m going to leave these sorts of optimizations for the next release.> > 3. Internal error: ERROR: "process_rule1" is not exported by the Shorewall::Rules module > > rules > ~~~~~ > SECTION RELATED > Related(IELOG(-,fw2NeT,2)) $FW net > > IELOG is inline action taking 3 parameters: (custom) disposition, (custom) chain and NFLOG class number. The full error message is: > ERROR: "process_rule1" is not exported by the Shorewall::Rules module > Can''t continue after import errors at /usr/share/shorewall/action.Related line 40 > BEGIN failed--compilation aborted at /usr/share/shorewall/action.Related line 40. > > I get the same error with "Established" and "Untracked". "Invalid" passesCrap -- wonder how that made it out the door. Removing this line from the failing actions will correct that issue: use Shorewall::Rules qw( process_rule1 );> though I have another 2 problems: > > -A +fw2net -m conntrack --ctstate INVALID -j LOG --log-tcp-options --log-ip-options --log-macdecode --log-tcp-sequence --log-uid --log-level 6 --log-prefix "Shorewall:fw2NeT::" > -A +fw2net -m conntrack --ctstate INVALID -j NFLOG --nflog-group 2 --nflog-range 0 --nflog-threshold 1 --nflog-prefix "Shorewall:fw2NeT::" > > Notice the "--cstate INVALID" matches - these could all be optimised in a single chain. I have tried custom actions with more statements (7 to be exact) and I had the same number of "--cstate INVALID" matches, which is wrong.So you believe that the compiler should somehow ignore ''inline'' and treat the action as if it were not inlined?> > I am ignoring the fact that in the RELATED state, the above statements will never execute (as the match is against INVALID), but this has already been raised. so I am not going to repeat it again.Again, optimization and detection of non-matching states will be left for another release.> > The second problem is this: > > rules > ~~~~~ > SECTION RELATED > IELOG(-,fw2NeT,2) > Invalid(IELOG(-,fw2NeT,2)) $FW net > > produces: > > -A +fw2net -m conntrack -j LOG --log-tcp-options --log-ip-options --log-macdecode --log-tcp-sequence --log-uid --log-level 6 --log-prefix "Shorewall:fw2NeT::" > -A +fw2net -m conntrack -j NFLOG --nflog-group 2 --nflog-range 0 --nflog-threshold 1 --nflog-prefix "Shorewall:fw2NeT::" > > In other words, not even a hint of "--cstate INVALID" (it could have been "optimised away" by mistake). In general, if the action above Invalid seems to be inline, the whole "Invalid(...)" statement seems to be totally ignored. > > 4. "normal" (not inline) actions now seem to be ignored by shorewall: > > actions > ~~~~~~~ > IELOG inline > ELOG > > Please note that I have action.ELOG symlinked to action.IELOG. In other words, these are exactly the same, only the definition in "actions" is different. > > rules > ~~~~~ > SECTION NEW > ELOG(-,fw2NeT,2) $FW net > [...] > > produces: > > -A fw2net -m conntrack --ctstate ESTABLISHED -j ACCEPT > -A fw2net -m conntrack --ctstate RELATED -j +fw2net > -A fw2net > [...] > > Note the last statement above - that won''t even compile! The same nonsensical statement is produced when I have the above ELOG statement placed in SECTION ALL. > > If I place ELOG in RELATED, that is completely ignored (as if it isn''t there). Same goes for: > - SECTION ESTABLISHED (the rule produced is "-A fw2net -m conntrack --ctstate ESTABLISHED"); > - SECTION UNTRACKED (the rule produced is "-A fw2net -m conntrack --ctstate UNTRACKED -j ~comb0" where "~comb0" consists of a single "-j DROP" statement); and > - SECTION INVALID (the rule produced is similar: "-A fw2net -m conntrack --ctstate INVALID -j ~comb0").Please post the action.IELOG file so I don''t have to guess what it does. Thanks, -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
On 02/02/2013 07:41 AM, Tom Eastep wrote:>> 1. Issue warning/error when state rules will never execute: >> >> rules >> ~~~~~ >> SECTION ESTABLISHED >> Invalid(DROP) all all >> >> or >> >> shorewall.conf >> ~~~~~~~~~~~~~~ >> BLACKLIST="NEW,UNTRACKED" >> >> blrules >> ~~~~~~~ >> Invalid(DROP) all all >> >> The above Invalid(DROP) rules will never execute since the outline state is NEW (rules) and "NEW,UNTRACKED" (blrules). > > Yes -- I admit that the compiler doesn''t try to understand unmatchable > rules; this is only one example of that lack.I''ve been able to pick some low-hanging fruit in this area; improvement will be seen in the next Beta/RC. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
On 02/02/2013 07:41 AM, Tom Eastep wrote:> On 02/01/2013 08:01 PM, Mr Dash Four wrote:>> actions >> ~~~~~~~ >> IELOG inline >> ELOG >> >> Please note that I have action.ELOG symlinked to action.IELOG. In other words, these are exactly the same, only the definition in "actions" is different. >> >> rules >> ~~~~~ >> SECTION NEW >> ELOG(-,fw2NeT,2) $FW net >> [...] >> >> produces: >> >> -A fw2net -m conntrack --ctstate ESTABLISHED -j ACCEPT >> -A fw2net -m conntrack --ctstate RELATED -j +fw2net >> -A fw2net >> [...] >> >> Note the last statement above - that won''t even compile! The same nonsensical statement is produced when I have the above ELOG statement placed in SECTION ALL. >> >> If I place ELOG in RELATED, that is completely ignored (as if it isn''t there). Same goes for: >> - SECTION ESTABLISHED (the rule produced is "-A fw2net -m conntrack --ctstate ESTABLISHED"); >> - SECTION UNTRACKED (the rule produced is "-A fw2net -m conntrack --ctstate UNTRACKED -j ~comb0" where "~comb0" consists of a single "-j DROP" statement); and >> - SECTION INVALID (the rule produced is similar: "-A fw2net -m conntrack --ctstate INVALID -j ~comb0"). > > Please post the action.IELOG file so I don''t have to guess what it does.Never mind -- I''ve been able to reproduce this; it is a consequence of setting @chain. Now to understand why.... -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
On 2/2/13 10:08 AM, "Tom Eastep" <teastep@shorewall.net> wrote:>On 02/02/2013 07:41 AM, Tom Eastep wrote: >> On 02/01/2013 08:01 PM, Mr Dash Four wrote: > >>> actions >>> ~~~~~~~ >>> IELOG inline >>> ELOG >>> >>> Please note that I have action.ELOG symlinked to action.IELOG. In >>>other words, these are exactly the same, only the definition in >>>"actions" is different. >>> >>> rules >>> ~~~~~ >>> SECTION NEW >>> ELOG(-,fw2NeT,2) $FW net >>> [...] >>> >>> produces: >>> >>> -A fw2net -m conntrack --ctstate ESTABLISHED -j ACCEPT >>> -A fw2net -m conntrack --ctstate RELATED -j +fw2net >>> -A fw2net >>> [...] >>> >>> Note the last statement above - that won''t even compile! The same >>>nonsensical statement is produced when I have the above ELOG statement >>>placed in SECTION ALL. >>> >>> If I place ELOG in RELATED, that is completely ignored (as if it isn''t >>>there). Same goes for: >>> - SECTION ESTABLISHED (the rule produced is "-A fw2net -m conntrack >>>--ctstate ESTABLISHED"); >>> - SECTION UNTRACKED (the rule produced is "-A fw2net -m conntrack >>>--ctstate UNTRACKED -j ~comb0" where "~comb0" consists of a single "-j >>>DROP" statement); and >>> - SECTION INVALID (the rule produced is similar: "-A fw2net -m >>>conntrack --ctstate INVALID -j ~comb0"). >> >> Please post the action.IELOG file so I don''t have to guess what it does. > >Never mind -- I''ve been able to reproduce this; it is a consequence of >setting @chain. Now to understand why....Here''s a patch. Thanks, -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
Tom In the attached config. rule: NotSyn(DROP) lan fw tcp 23 produces the following error message: ERROR: SOURCE/DEST PORT(S) not allowed without PROTO /usr/share/shorewall/action.NotSyn (line 61) from /etc/shorewall2A19/rules (line 23) If I change PROTO to UDP or NotSyn to noinline the problem does not occur. Steven. ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
On 2/2/13 11:08 AM, "Steven Jan Springl" <steven@springl.ukfsn.org> wrote:>Tom > >In the attached config. rule: > >NotSyn(DROP) lan fw tcp 23 > >produces the following error message: > >ERROR: SOURCE/DEST PORT(S) not allowed without PROTO >/usr/share/shorewall/action.NotSyn (line 61) > from /etc/shorewall2A19/rules (line 23) > >If I change PROTO to UDP or NotSyn to noinline the problem does not occur.The attached patch corrects this issue. Thanks Steven, -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
On Saturday 02 Feb 2013 20:16:29 Tom Eastep wrote:> On 2/2/13 11:08 AM, "Steven Jan Springl" <steven@springl.ukfsn.org> wrote: > >Tom > > > >In the attached config. rule: > > > >NotSyn(DROP) lan fw tcp 23 > > > >produces the following error message: > > > >ERROR: SOURCE/DEST PORT(S) not allowed without PROTO > >/usr/share/shorewall/action.NotSyn (line 61) > > > > from /etc/shorewall2A19/rules (line 23) > > > >If I change PROTO to UDP or NotSyn to noinline the problem does not occur. > > The attached patch corrects this issue. > > Thanks Steven, > > -Tom > You do not need a parachute to skydive. You only need a parachute to > skydive twice.Tom Confirmed, the patch corrects the issue. Thanks. Steven. ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
On 2/2/13 1:12 PM, "Steven Jan Springl" <steven@springl.ukfsn.org> wrote:> >Confirmed, the patch corrects the issue. >Thanks Steven, -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
>>> Please post the action.IELOG file so I don''t have to guess what it does.I described the function of that action in my post, but missed the most important part (which was ultimately the cause of this problem): The action takes 3 parameters: new (custom) disposition, new (custom) chain and NFLOG class. It creates one LOG and one NFLOG target statements. The action is not inline, though its "counterpart" - IELOG - is.>> Never mind -- I''ve been able to reproduce this; it is a consequence of >> setting @chain. Now to understand why.... > > Here''s a patch.Yep, that does the trick. However, I have found another issue: rules ~~~~~ SECTION ESTABLISHED IELOG(-,fw2NeT,2) $FW net ACCEPT $FW net udp ACCEPT $FW net tcp produces this: -A fw2net -m conntrack --ctstate ESTABLISHED -j LOG --log-tcp-options --log-ip-options --log-macdecode --log-tcp-sequence --log-uid --log-level 6 --log-prefix "Shorewall:fw2NeT::" -A fw2net -m conntrack --ctstate ESTABLISHED -j NFLOG --nflog-group 2 --nflog-range 0 --nflog-threshold 1 --nflog-prefix "Shorewall:fw2NeT::" -A fw2net -p 17 -m conntrack --ctstate ESTABLISHED -j ACCEPT -A fw2net -p 6 -m conntrack --ctstate ESTABLISHED -j ACCEPT -A fw2net -m conntrack --ctstate ESTABLISHED -j ACCEPT As evident, again, there is no optimisation on the "--cstate" at all. All the ESTABLISHED states should have been optimised away in a separate chain with a single --cstate match. ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
> Crap -- wonder how that made it out the door. > > Removing this line from the failing actions will correct that issue: > > use Shorewall::Rules qw( process_rule1 );It does, though there is another issue: rules ~~~~~ SECTION RELATED Related(ELOG(-,fw2NeT,2)) $FW net produces: -A +fw2net -m conntrack --ctstate RELATED -j ELOG "--cstate RELATED" match can be optimised away (it is not needed since the +fw2net chain has that match already). The "inline" equivalent of ELOG (IELOG) produces 2 additional RELATED matches (for each statement of that action) as well, but I suspect you already know that. I also suspect the situation will be the same if I use Established in the ESTABLISHED section, Untracked in the UNTRACKED section and Invalid in the INVALID section.> So you believe that the compiler should somehow ignore ''inline'' and > treat the action as if it were not inlined?Yeah, getting ahead of myself. You are right there.> Again, optimization and detection of non-matching states will be left > for another release.Fair enough, as long as I am aware of these deficiencies it is all fine by me.> >> The second problem is this: >> >> rules >> ~~~~~ >> SECTION RELATED >> IELOG(-,fw2NeT,2) >> Invalid(IELOG(-,fw2NeT,2)) $FW net >> >> produces: >> >> -A +fw2net -m conntrack -j LOG --log-tcp-options --log-ip-options --log-macdecode --log-tcp-sequence --log-uid --log-level 6 --log-prefix "Shorewall:fw2NeT::" >> -A +fw2net -m conntrack -j NFLOG --nflog-group 2 --nflog-range 0 --nflog-threshold 1 --nflog-prefix "Shorewall:fw2NeT::" >> >> In other words, not even a hint of "--cstate INVALID" (it could have been "optimised away" by mistake). In general, if the action above Invalid seems to be inline, the whole "Invalid(...)" statement seems to be totally ignored.Anything on this? ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
> I''ve been able to pick some low-hanging fruit in this area; improvement > will be seen in the next Beta/RC.Now that you''ve split up and implemented the various states in separate chains, could you not keep a track of these when each chain is processed and if there is a different "--cstate" match, then produce a warning and remove that statement and if there is an extra "--cstate" match (like --cstate INVALID in the "invalid" chain) then just remove that extra match. ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
On 02/02/2013 05:58 PM, Mr Dash Four wrote:> >>>> Please post the action.IELOG file so I don''t have to guess what it does. > I described the function of that action in my post, but missed the most important part (which was ultimately the cause of this problem): The action takes 3 parameters: new (custom) disposition, new (custom) chain and NFLOG class. It creates one LOG and one NFLOG target statements. The action is not inline, though its "counterpart" - IELOG - is. > >>> Never mind -- I''ve been able to reproduce this; it is a consequence of >>> setting @chain. Now to understand why.... >> >> Here''s a patch. > Yep, that does the trick. However, I have found another issue: > > rules > ~~~~~ > SECTION ESTABLISHED > IELOG(-,fw2NeT,2) $FW net > ACCEPT $FW net udp > ACCEPT $FW net tcp > > produces this: > > -A fw2net -m conntrack --ctstate ESTABLISHED -j LOG --log-tcp-options --log-ip-options --log-macdecode --log-tcp-sequence --log-uid --log-level 6 --log-prefix "Shorewall:fw2NeT::" > -A fw2net -m conntrack --ctstate ESTABLISHED -j NFLOG --nflog-group 2 --nflog-range 0 --nflog-threshold 1 --nflog-prefix "Shorewall:fw2NeT::" > -A fw2net -p 17 -m conntrack --ctstate ESTABLISHED -j ACCEPT > -A fw2net -p 6 -m conntrack --ctstate ESTABLISHED -j ACCEPT > -A fw2net -m conntrack --ctstate ESTABLISHED -j ACCEPT > > As evident, again, there is no optimisation on the "--cstate" at all. All the ESTABLISHED states should have been optimised away in a separate chain with a single --cstate match.I haven''t place ESTABLISHED in it''s own chain, primarily because I haven''t found a use case for ESTABLISHED rules at all. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
On 02/02/2013 05:59 PM, Mr Dash Four wrote:> >> Crap -- wonder how that made it out the door. >> >> Removing this line from the failing actions will correct that >> issue: >> >> use Shorewall::Rules qw( process_rule1 ); > It does, though there is another issue: > > rules ~~~~~ SECTION RELATED Related(ELOG(-,fw2NeT,2)) $FW net > > produces: > > -A +fw2net -m conntrack --ctstate RELATED -j ELOG > > "--cstate RELATED" match can be optimised away (it is not needed > since the +fw2net chain has that match already). The "inline" > equivalent of ELOG (IELOG) produces 2 additional RELATED matches (for > each statement of that action) as well, but I suspect you already > know that. I also suspect the situation will be the same if I use > Established in the ESTABLISHED section, Untracked in the UNTRACKED > section and Invalid in the INVALID section.All of that is corrected in my current tree.> >> So you believe that the compiler should somehow ignore ''inline'' >> and treat the action as if it were not inlined? > Yeah, getting ahead of myself. You are right there. > >> Again, optimization and detection of non-matching states will be >> left for another release. > Fair enough, as long as I am aware of these deficiencies it is all > fine by me.As I mentioned in a later post yesterday, I''ve been able to do some work in that area for this release.> >> >>> The second problem is this: >>> >>> rules ~~~~~ SECTION RELATED IELOG(-,fw2NeT,2) >>> Invalid(IELOG(-,fw2NeT,2)) $FW net >>> >>> produces: >>> >>> -A +fw2net -m conntrack -j LOG --log-tcp-options --log-ip-options >>> --log-macdecode --log-tcp-sequence --log-uid --log-level 6 >>> --log-prefix "Shorewall:fw2NeT::" -A +fw2net -m conntrack -j >>> NFLOG --nflog-group 2 --nflog-range 0 --nflog-threshold 1 >>> --nflog-prefix "Shorewall:fw2NeT::" >>> >>> In other words, not even a hint of "--cstate INVALID" (it could >>> have been "optimised away" by mistake). In general, if the action >>> above Invalid seems to be inline, the whole "Invalid(...)" >>> statement seems to be totally ignored. > Anything on this?I posted a patch yesterday. Cheers, -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
On 02/02/2013 05:59 PM, Mr Dash Four wrote:> >> I''ve been able to pick some low-hanging fruit in this area; >> improvement will be seen in the next Beta/RC. > Now that you''ve split up and implemented the various states in > separate chains, could you not keep a track of these when each chain > is processed and if there is a different "--cstate" match, then > produce a warning and remove that statement and if there is an extra > "--cstate" match (like --cstate INVALID in the "invalid" chain) then > just remove that extra match.Already done. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan
> I haven''t place ESTABLISHED in it''s own chain, primarily because I > haven''t found a use case for ESTABLISHED rules at all.Bandwidth traffic limiting (in other words, throttling), traffic sniffing/logging/manipulating (pcap and friends). For the latter, one particularly good example is sip-related traffic (manipulating the SIP header, primarily). I could think of other uses if you are so inclined... If there is no "use case" for ESTABLISHED, then why is it present in "rules"? One expects to add rules there and have these properly optimised - the way any other state is (or should be). Currently, this isn''t the case (even in RC2). ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
>> rules ~~~~~ SECTION RELATED Related(ELOG(-,fw2NeT,2)) $FW net >> >> produces: >> >> -A +fw2net -m conntrack --ctstate RELATED -j ELOG >> >> "--cstate RELATED" match can be optimised away (it is not needed >> since the +fw2net chain has that match already). The "inline" >> equivalent of ELOG (IELOG) produces 2 additional RELATED matches (for >> each statement of that action) as well, but I suspect you already >> know that. I also suspect the situation will be the same if I use >> Established in the ESTABLISHED section, Untracked in the UNTRACKED >> section and Invalid in the INVALID section. > > All of that is corrected in my current tree.This now works. However: rules ~~~~~ SECTION RELATED Established(ELOG(-,fw2NeT,2)) $FW net Established(dropInvalid) $FW net dropInvalid $FW net produces: -A +fw2net -m conntrack --ctstate ESTABLISHED -j ELOG -A +fw2net -m conntrack --ctstate ESTABLISHED -m conntrack --ctstate INVALID -j DROP -A +fw2net -m conntrack --ctstate INVALID -j DROP "ELOG" is not inline and is the equivalent of IELOG as indicated previously. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
> This now works. However: > > rules > ~~~~~ > SECTION RELATED > Established(ELOG(-,fw2NeT,2)) $FW net > Established(dropInvalid) $FW net > dropInvalid $FW net > > produces: > > -A +fw2net -m conntrack --ctstate ESTABLISHED -j ELOG > -A +fw2net -m conntrack --ctstate ESTABLISHED -m conntrack --ctstate INVALID -j DROP > -A +fw2net -m conntrack --ctstate INVALID -j DROP > > "ELOG" is not inline and is the equivalent of IELOG as indicated previously.That now also works as expected. However: rules ~~~~~ SECTION RELATED Related(ELOG(,fw2NeT,2)) $FW net DROP $FW net udp DROP $FW net tcp produces: -A +fw2net -p 17 -j DROP -A +fw2net -p 6 -j DROP -A +fw2net -j ACCEPT In other words, the "Related" action is now "optimised" away for some reason. If I use "ELOG(,fw2NeT,2)" directly, instead of "Related(ELOG(,fw2NeT,2))" that works as expected. Please also note that I have RELATED_DISPOSITION=ACCEPT in shorewall.conf. If I had a warning of whether the "wrong" rules (in shorewall''s opinion) have been ignored/optimised, I would have known whether this is ignored deliberately or whether there is something wrong with "Related" when specifying custom action as a parameter. As it stands, I can''t tell as shorewall is silent. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/9/13 10:44 AM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:>That now also works as expected. However: > >rules >~~~~~ >SECTION RELATED >Related(ELOG(,fw2NeT,2)) $FW net >DROP $FW net udp >DROP $FW net tcp > >produces: > >-A +fw2net -p 17 -j DROP >-A +fw2net -p 6 -j DROP >-A +fw2net -j ACCEPT > >In other words, the "Related" action is now "optimised" away for some >reason. If I use "ELOG(,fw2NeT,2)" directly, instead of >"Related(ELOG(,fw2NeT,2))" that works as expected. Please also note that >I have RELATED_DISPOSITION=ACCEPT in shorewall.conf.Simple typo -- I typed "$_" when I wanted "$1". Patch attached.> > >If I had a warning of whether the "wrong" rules (in shorewall''s opinion) >have been ignored/optimised, I would have known whether this is ignored >deliberately or whether there is something wrong with "Related" when >specifying custom action as a parameter. As it stands, I can''t tell as >shorewall is silent.You should be seeing messages such as these when a rule is suppressed: Checking /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules... Checking /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/action.ELOG for chain ELOG... WARNING: Entry generated no iptables rules /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line 22) WARNING: Entry generated no iptables rules /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line 24) WARNING: Entry generated no iptables rules /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line 32) Checking /usr/share/shorewall/action.TCPFlags for chain TCPFlags... Checking /usr/share/shorewall/action.Broadcast for chain Broadcast... -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
> Simple typo -- I typed "$_" when I wanted "$1". Patch attached.That file is empty.> You should be seeing messages such as these when a rule is suppressed: > > Checking /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules... > Checking > /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/action.ELOG for > chain ELOG... > WARNING: Entry generated no iptables rules > /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line 22) > WARNING: Entry generated no iptables rules > /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line 24) > WARNING: Entry generated no iptables rules > /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line 32) > Checking /usr/share/shorewall/action.TCPFlags for chain TCPFlags... > Checking /usr/share/shorewall/action.Broadcast for chain Broadcast...Nope, I don''t see any of these. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/9/13 11:17 AM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:> >> Simple typo -- I typed "$_" when I wanted "$1". Patch attached. >That file is empty.Here''s the correct file.> >> You should be seeing messages such as these when a rule is suppressed: >> >> Checking >>/home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules... >> Checking >> /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/action.ELOG for >> chain ELOG... >> WARNING: Entry generated no iptables rules >> /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line 22) >> WARNING: Entry generated no iptables rules >> /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line 24) >> WARNING: Entry generated no iptables rules >> /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line 32) >> Checking /usr/share/shorewall/action.TCPFlags for chain TCPFlags... >> Checking /usr/share/shorewall/action.Broadcast for chain Broadcast... >Nope, I don''t see any of these.Interesting. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
> Here''s the correct file.Yep, that now works.>> Nope, I don''t see any of these. > > Interesting.My shorewall.conf settings? ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/9/13 11:40 AM, "Tom Eastep" <teastep@shorewall.net> wrote:>> >>> You should be seeing messages such as these when a rule is suppressed: >>> >>> Checking >>>/home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules... >>> Checking >>> /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/action.ELOG >>>for >>> chain ELOG... >>> WARNING: Entry generated no iptables rules >>> /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line >>>22) >>> WARNING: Entry generated no iptables rules >>> /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line >>>24) >>> WARNING: Entry generated no iptables rules >>> /home/teastep/shorewall/regressionLibrary/4.5.13/inline6/rules (line >>>32) >>> Checking /usr/share/shorewall/action.TCPFlags for chain TCPFlags... >>> Checking /usr/share/shorewall/action.Broadcast for chain Broadcast... >>Nope, I don''t see any of these. > >Interesting.Please forward a configuration that results in rules being dropped silently and I''ll take a look. Thanks, -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
> Please forward a configuration that results in rules being dropped > silently and I''ll take a look.I''ll try and prepare a "minimalistic" configuration for you (with as little clutter as possible) later on. What could be the suspect cause for this - apart from rules and shorewall.conf? ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/9/13 4:40 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:>> Please forward a configuration that results in rules being dropped >> silently and I''ll take a look. >I''ll try and prepare a "minimalistic" configuration for you (with as >little clutter as possible) later on. What could be the suspect cause for >this - apart from rules and shorewall.conf?Rules, shorewall.conf, actions and blrules if the problem is occurring there. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
>>> Please forward a configuration that results in rules being dropped >>> silently and I''ll take a look. >>> >> I''ll try and prepare a "minimalistic" configuration for you (with as >> little clutter as possible) later on. What could be the suspect cause for >> this - apart from rules and shorewall.conf? >> > > Rules, shorewall.conf, actions and blrules if the problem is occurring > there. >Are you still interested in this, or should I wait until the next release? ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/10/13 3:38 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:> >>>> Please forward a configuration that results in rules being dropped >>>> silently and I''ll take a look. >>>> >>> I''ll try and prepare a "minimalistic" configuration for you (with as >>> little clutter as possible) later on. What could be the suspect cause >>>for >>> this - apart from rules and shorewall.conf? >>> >> >> Rules, shorewall.conf, actions and blrules if the problem is occurring >> there. >> >Are you still interested in this, or should I wait until the next release?If you have it, go ahead and forward it and I''ll look at it this evening. Otherwise, let''s wait for 4.5.14 Beta 1 (which will be released shortly after 4.5.13). -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
> If you have it, go ahead and forward it and I''ll look at it this evening. > Otherwise, let''s wait for 4.5.14 Beta 1 (which will be released shortly > after 4.5.13). >No, I don''t have it (well, I have the whole lot, but need to weed it out and then send you the relevant parts - all this is on a separate machine), but can prepare it in about 3 hours. What would you like me to do? ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/10/13 3:50 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:> >> If you have it, go ahead and forward it and I''ll look at it this >>evening. >> Otherwise, let''s wait for 4.5.14 Beta 1 (which will be released shortly >> after 4.5.13). >> >No, I don''t have it (well, I have the whole lot, but need to weed it out >and then send you the relevant parts - all this is on a separate >machine), but can prepare it in about 3 hours. What would you like me to >do?Just prepare it as you have time and I''ll look at it. I don''t think it is critical to include a change in this release. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
> Just prepare it as you have time and I''ll look at it. I don''t think it is > critical to include a change in this release.The more I dig into this, the more idiosyncrasies I find... When I try and copy my (test case) shorewall.conf, actions, rules, blrules, and even shorewallrc files into a separate directory and then try to execute "shorewall compile firewall" I get this: ERROR: Ordinary users may not compile the /etc/shorewall configuration [Oh, really?] My shorewall.conf *explicitly* states that "CONFIG_PATH=.:${SHAREDIR}/shorewall" so why the hell is shorewall trying to access /etc/shorewall? Even if I move my own copy of shorewallrc to the test case directory and amend CONFDIR to point to my own (test case) directory, this has no effect whatsoever. When I execute "shorewall compile . firewall", this time I get "/usr/share/shorewall/lib.cli-std: line 41: /etc/shorewall/params: Permission denied". Again, why the hell is shorewall looking in /etc/shorewall? lib.cli-std uses "find_file", which is in lib.common and when I do a quick hack into this function, I see that ${CONFDIR} is actually "/etc" and {CONFIG_PATH} is "<my test case dir>:/etc/:/usr/share/shorewall". So, why is shorewall disregarding my own shorewallrc, then "constructs" some mickey-mouse CONFIG_PATH which has absolutely nothing whatsoever to do with the one I specified in my (test case) shorewall.conf? Executing "shorewall compile -e . filrewall" or "shorewall compile -e firewall" is no good either. How am I supposed to compile a shorewall configuration independent of the host system then, using my own test case directory and the CONFIG_PATH I have specified in my own shorewall.conf? I was able to do this before, but now, it seems, this has been messed up again. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/10/13 6:58 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:>> Just prepare it as you have time and I''ll look at it. I don''t think it >>is >> critical to include a change in this release. >The more I dig into this, the more idiosyncrasies I find... > >When I try and copy my (test case) shorewall.conf, actions, rules, >blrules, and even shorewallrc files into a separate directory and then >try to execute "shorewall compile firewall" I get this: > >ERROR: Ordinary users may not compile the /etc/shorewall configuration >[Oh, really?]Shorewall compile <directory> <script>> >My shorewall.conf *explicitly* states that >"CONFIG_PATH=.:${SHAREDIR}/shorewall" so why the hell is shorewall trying >to access /etc/shorewall? Even if I move my own copy of shorewallrc to >the test case directory and amend CONFDIR to point to my own (test case) >directory, this has no effect whatsoever. > >When I execute "shorewall compile . firewall", this time I get >"/usr/share/shorewall/lib.cli-std: line 41: /etc/shorewall/params: >Permission denied". > >Again, why the hell is shorewall looking in /etc/shorewall? lib.cli-std >uses "find_file", which is in lib.common and when I do a quick hack into >this function, I see that ${CONFDIR} is actually "/etc" and {CONFIG_PATH} >is "<my test case dir>:/etc/:/usr/share/shorewall". > >So, why is shorewall disregarding my own shorewallrc, then "constructs" >some mickey-mouse CONFIG_PATH which has absolutely nothing whatsoever to >do with the one I specified in my (test case) shorewall.conf? > >Executing "shorewall compile -e . filrewall" or "shorewall compile -e >firewall" is no good either. > >How am I supposed to compile a shorewall configuration independent of the >host system then, using my own test case directory and the CONFIG_PATH I >have specified in my own shorewall.conf? I was able to do this before, >but now, it seems, this has been messed up again.Try reading the documentation? -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
> Shorewall compile <directory> <script>It is actually "shorewall compile [directory] [pathname]" according to your own man page (with the latter two parameters being optional).> >> My shorewall.conf *explicitly* states that >> "CONFIG_PATH=.:${SHAREDIR}/shorewall" so why the hell is shorewall trying >> to access /etc/shorewall? Even if I move my own copy of shorewallrc to >> the test case directory and amend CONFDIR to point to my own (test case) >> directory, this has no effect whatsoever. >> >> When I execute "shorewall compile . firewall", this time I get >> "/usr/share/shorewall/lib.cli-std: line 41: /etc/shorewall/params: >> Permission denied". >> >> Again, why the hell is shorewall looking in /etc/shorewall? lib.cli-std >> uses "find_file", which is in lib.common and when I do a quick hack into >> this function, I see that ${CONFDIR} is actually "/etc" and {CONFIG_PATH} >> is "<my test case dir>:/etc/:/usr/share/shorewall". >> >> So, why is shorewall disregarding my own shorewallrc, then "constructs" >> some mickey-mouse CONFIG_PATH which has absolutely nothing whatsoever to >> do with the one I specified in my (test case) shorewall.conf? >> >> Executing "shorewall compile -e . filrewall" or "shorewall compile -e >> firewall" is no good either. >> >> How am I supposed to compile a shorewall configuration independent of the >> host system then, using my own test case directory and the CONFIG_PATH I >> have specified in my own shorewall.conf? I was able to do this before, >> but now, it seems, this has been messed up again. > > Try reading the documentation?And? Don''t you think that I''ve already done that before posting? What exactly is it that I am missing - care to point out? ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/10/13 7:05 PM, "Tom Eastep" <teastep@shorewall.net> wrote:>On 2/10/13 6:58 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote: > >>> Just prepare it as you have time and I''ll look at it. I don''t think it >>>is >>> critical to include a change in this release. >>The more I dig into this, the more idiosyncrasies I find... >> >>When I try and copy my (test case) shorewall.conf, actions, rules, >>blrules, and even shorewallrc files into a separate directory and then >>try to execute "shorewall compile firewall" I get this: >> >>ERROR: Ordinary users may not compile the /etc/shorewall configuration >>[Oh, really?] > >Shorewall compile <directory> <script> > >> >>My shorewall.conf *explicitly* states that >>"CONFIG_PATH=.:${SHAREDIR}/shorewall" so why the hell is shorewall trying >>to access /etc/shorewall? Even if I move my own copy of shorewallrc to >>the test case directory and amend CONFDIR to point to my own (test case) >>directory, this has no effect whatsoever. >> >>When I execute "shorewall compile . firewall", this time I get >>"/usr/share/shorewall/lib.cli-std: line 41: /etc/shorewall/params: >>Permission denied". >> >>Again, why the hell is shorewall looking in /etc/shorewall? lib.cli-std >>uses "find_file", which is in lib.common and when I do a quick hack into >>this function, I see that ${CONFDIR} is actually "/etc" and {CONFIG_PATH} >>is "<my test case dir>:/etc/:/usr/share/shorewall". >> >>So, why is shorewall disregarding my own shorewallrc, then "constructs" >>some mickey-mouse CONFIG_PATH which has absolutely nothing whatsoever to >>do with the one I specified in my (test case) shorewall.conf? >> >>Executing "shorewall compile -e . filrewall" or "shorewall compile -e >>firewall" is no good either. >> >>How am I supposed to compile a shorewall configuration independent of the >>host system then, using my own test case directory and the CONFIG_PATH I >>have specified in my own shorewall.conf? I was able to do this before, >>but now, it seems, this has been messed up again. > >Try reading the documentation?Another note -- when someone sends me a problem configuration, the *first* thing I do is edit the shorewall.conf file and set CONFIG_PATH=/usr/share/shorewall/ (that''s where I install Shorewall --YMMV). That stops the compiler from looking in /etc/shorewall. Now I can compile the configuration without interference from /etc/shorewall. The *second* thing I do is add a capabilities file if the person neglected to include one. Now I can ''shorewall check .'' and ''shorewall compile . firewall'' and analyze the problem. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
> Another note -- when someone sends me a problem configuration, the *first* > thing I do is edit the shorewall.conf file and set > CONFIG_PATH=/usr/share/shorewall/ (that''s where I install Shorewall > --YMMV).As I already pointed out, my "CONFIG_PATH=.:/usr/share/shorewall" (that''s the config path in shorewall.conf which resides in my test case directory), so I don''t see why shorewall decided not to honour this setting and started looking in /etc/shorewall for whatever reason. Besides, setting "CONFIG_PATH=/usr/share/shorewall" in shorewall.conf still gets me "/usr/share/shorewall/lib.cli-std: line 41: /etc/shorewall/params: Permission denied".> That stops the compiler from looking in /etc/shorewall. Now I can > compile the configuration without interference from /etc/shorewall.That still doesn''t explain why lib.common has CONFIG_PATH set to different values (including shoving "/etc" in there) to what I have specified in my own shorewall.conf, which was more or less the point I was making.> The > *second* thing I do is add a capabilities file if the person neglected to > include one. Now I can ''shorewall check .'' and ''shorewall compile . > firewall'' and analyze the problem.I haven''t reached that far - one problem at the time. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/10/13 7:30 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:> >> Another note -- when someone sends me a problem configuration, the >>*first* >> thing I do is edit the shorewall.conf file and set >> CONFIG_PATH=/usr/share/shorewall/ (that''s where I install Shorewall >> --YMMV). >As I already pointed out, my "CONFIG_PATH=.:/usr/share/shorewall" (that''s >the config path in shorewall.conf which resides in my test case >directory), so I don''t see why shorewall decided not to honour this >setting and started looking in /etc/shorewall for whatever reason. > >Besides, setting "CONFIG_PATH=/usr/share/shorewall" in shorewall.conf >still gets me "/usr/share/shorewall/lib.cli-std: line 41: >/etc/shorewall/params: Permission denied".That looks like /etc/shorewall is still in the CONFIG_PATH. I wouldn''t be able to support Shorewall at all if this problem was ubiquitous. So something in your config is directing the compiler to /etc/shorewall. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
>>> Another note -- when someone sends me a problem configuration, the >>> *first* >>> thing I do is edit the shorewall.conf file and set >>> CONFIG_PATH=/usr/share/shorewall/ (that''s where I install Shorewall >>> --YMMV). >> As I already pointed out, my "CONFIG_PATH=.:/usr/share/shorewall" (that''s >> the config path in shorewall.conf which resides in my test case >> directory), so I don''t see why shorewall decided not to honour this >> setting and started looking in /etc/shorewall for whatever reason. >> >> Besides, setting "CONFIG_PATH=/usr/share/shorewall" in shorewall.conf >> still gets me "/usr/share/shorewall/lib.cli-std: line 41: >> /etc/shorewall/params: Permission denied". > > That looks like /etc/shorewall is still in the CONFIG_PATH.Whatever it is, it is not from my shorewall.conf as evident from what I posted above. The question is - if it isn''t my own shorewall.conf, what is it? Whatever it is, or whatever shorewall is doing in not honouring either that config path or the path specified on the command line ("shorewall compile -e . firewall"), it is wrong.> I wouldn''t be able to support Shorewall at all if this problem was > ubiquitous. So something in your config is directing the compiler to > /etc/shorewall.If you can''t tell what it is (and you wrote that software), then what hope is left for any of us? If it isn''t shorewall.conf, and it isn''t the path specified on the command line either, then what is it? ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/10/13 8:26 PM, "Mr Dash Four" <mr.dash.four@googlemail.com> wrote:> >>>> Another note -- when someone sends me a problem configuration, the >>>> *first* >>>> thing I do is edit the shorewall.conf file and set >>>> CONFIG_PATH=/usr/share/shorewall/ (that''s where I install Shorewall >>>> --YMMV). >>> As I already pointed out, my "CONFIG_PATH=.:/usr/share/shorewall" >>>(that''s >>> the config path in shorewall.conf which resides in my test case >>> directory), so I don''t see why shorewall decided not to honour this >>> setting and started looking in /etc/shorewall for whatever reason. >>> >>> Besides, setting "CONFIG_PATH=/usr/share/shorewall" in shorewall.conf >>> still gets me "/usr/share/shorewall/lib.cli-std: line 41: >>> /etc/shorewall/params: Permission denied". >> >> That looks like /etc/shorewall is still in the CONFIG_PATH. >Whatever it is, it is not from my shorewall.conf as evident from what I >posted above. The question is - if it isn''t my own shorewall.conf, what >is it? Whatever it is, or whatever shorewall is doing in not honouring >either that config path or the path specified on the command line >("shorewall compile -e . firewall"), it is wrong. > >> I wouldn''t be able to support Shorewall at all if this problem was >> ubiquitous. So something in your config is directing the compiler to >> /etc/shorewall. >If you can''t tell what it is (and you wrote that software), then what >hope is left for any of us? If it isn''t shorewall.conf, and it isn''t the >path specified on the command line either, then what is it?The answer is on your computer, not mine. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
>>>>> Another note -- when someone sends me a problem configuration, the >>>>> *first* >>>>> thing I do is edit the shorewall.conf file and set >>>>> CONFIG_PATH=/usr/share/shorewall/ (that''s where I install Shorewall >>>>> --YMMV). >>>> As I already pointed out, my "CONFIG_PATH=.:/usr/share/shorewall" >>>> (that''s >>>> the config path in shorewall.conf which resides in my test case >>>> directory), so I don''t see why shorewall decided not to honour this >>>> setting and started looking in /etc/shorewall for whatever reason. >>>> >>>> Besides, setting "CONFIG_PATH=/usr/share/shorewall" in shorewall.conf >>>> still gets me "/usr/share/shorewall/lib.cli-std: line 41: >>>> /etc/shorewall/params: Permission denied". >>> That looks like /etc/shorewall is still in the CONFIG_PATH. >> Whatever it is, it is not from my shorewall.conf as evident from what I >> posted above. The question is - if it isn''t my own shorewall.conf, what >> is it? Whatever it is, or whatever shorewall is doing in not honouring >> either that config path or the path specified on the command line >> ("shorewall compile -e . firewall"), it is wrong. >> >>> I wouldn''t be able to support Shorewall at all if this problem was >>> ubiquitous. So something in your config is directing the compiler to >>> /etc/shorewall. >> If you can''t tell what it is (and you wrote that software), then what >> hope is left for any of us? If it isn''t shorewall.conf, and it isn''t the >> path specified on the command line either, then what is it? > > The answer is on your computer, not mine.Which is what exactly (provided you know it, that is)? ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 2/10/13 8:38 PM, "Tom Eastep" <teastep@shorewall.net> wrote:> >The answer is on your computer, not mine.But if you send me a tarball, then maybe the answer will be on my computer. -Tom You do not need a parachute to skydive. You only need a parachute to skydive twice. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
>> The answer is on your computer, not mine. > > But if you send me a tarball, then maybe the answer will be on my computer.I am sorry, but I don''t play hide-and-seek - if you know what else could have caused this, I am a willing participant. If you don''t know (or won''t tell), then I don''t have time to waste. I was willing to help you out, but if you can''t be arsed, then so be it. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 02/10/2013 08:50 PM, Mr Dash Four wrote:> >>> The answer is on your computer, not mine. >> >> But if you send me a tarball, then maybe the answer will be on my >> computer. > I am sorry, but I don''t play hide-and-seek - if you know what else > could have caused this, I am a willing participant. If you don''t know > (or won''t tell), then I don''t have time to waste. I was willing to > help you out, but if you can''t be arsed, then so be it.Well, the most obvious answer is that there is no params file in the directory that you are trying to compile. /sbin/shorewall reads shorewall.conf prior to launching the compiler. Given that shorewall.conf can use variables from params, the params file is read first. Because shorewall.conf has not be read yet, its CONFIG_PATH setting is not yet available. The CONFIG_PATH used is formed by prepending the directory named in the compile or check command, to the default CONFIG_PATH. So, at a minimum, the directory named in a ''compile'' or ''check'' command must contain a params file and shorewall.conf. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 02/11/2013 05:52 AM, Tom Eastep wrote:> On 02/10/2013 08:50 PM, Mr Dash Four wrote: >> >>>> The answer is on your computer, not mine. >>> >>> But if you send me a tarball, then maybe the answer will be on my >>> computer. >> I am sorry, but I don''t play hide-and-seek - if you know what else >> could have caused this, I am a willing participant. If you don''t know >> (or won''t tell), then I don''t have time to waste. I was willing to >> help you out, but if you can''t be arsed, then so be it. > > Well, the most obvious answer is that there is no params file in the > directory that you are trying to compile. > > /sbin/shorewall reads shorewall.conf prior to launching the compiler. > Given that shorewall.conf can use variables from params, the params file > is read first. Because shorewall.conf has not be read yet, its > CONFIG_PATH setting is not yet available. > > The CONFIG_PATH used is formed by prepending the directory named in the > compile or check command, to the default CONFIG_PATH. > > So, at a minimum, the directory named in a ''compile'' > or ''check'' command must contain a params file and shorewall.conf. >I plan to leave this as it is for 4.5.13. In 4.5.14, though, I could change the code so that: a) If the user is root, the behavior remains the same. b) If the user is not root, /sbin/shorewall (actulally lib.cli-std) would only look for the params file in the directory specified on the command line. I prefer the current behavior for root, because it allows for creation of a temporary test directory containing only those configuration files that are being changed. It is not necessary that the directory contain params and shorewall.conf. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
> Well, the most obvious answer is that there is no params file in the > directory that you are trying to compile. >That is not really the issue, is it? The real issue is that for some reason shorewall is attempting access to "/etc" (and, by extension, "/etrc/shorewall") without me asking (or granting) shorewall such access.> /sbin/shorewall reads shorewall.conf prior to launching the compiler. >So? Again, that is not the issue - the issue is ... see above, I can''t be bothered.> Given that shorewall.conf can use variables from params, the params file > is read first.And?> Because shorewall.conf has not be read yet, its > CONFIG_PATH setting is not yet available. > > The CONFIG_PATH used is formed by prepending the directory named in the > compile or check command, to the default CONFIG_PATH. > > So, at a minimum, the directory named in a ''compile'' > or ''check'' command must contain a params file and shorewall.conf. >I''ve never said/nor implied that the presence/absence of "params" (or any other) file is the issue. It isn''t! For the last time - why is shorewall reading/attempting to access my "/etc" directory (and, by extension, "/etc/shorewall"), given that I have not specified, nor desire, nor granted shorewall such access? Second question: why is shorewall modifying my CONFIG_PATH to include "/etc" (and, by extension, "/etc/shorewall") - I''ve never specified such directory, nor desired (or granted) shorewall such access when executing "shorewall compile . firewall" to produce a separate-and-nothing-to-do-with-my-host-shorewall-configuration firewall script? If you are still struggling to comprehend the above points, here is the short version - I have no problems with shorewall screaming at me if/when "params" (or anything else) is missing - what I have an issue with is that shorewall, for whatever reason, unilaterally decided to access "/etc" (and, by extension, "/etc/shorewall") and place that directory in the middle of my CONFIG_PATH variable, without my knowledge or consent, screwing everything up in the process. The access to /"etc" was restricted by myself, deliberately, so that I could catch shorewall with its pants down (as evident from the above) if it attempts such access when none is desired, when I produce my shorewall firewall script designed to be used elsewhere. ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb
On 02/13/2013 05:46 AM, Mr Dash Four wrote:> >> Well, the most obvious answer is that there is no params file in the >> directory that you are trying to compile. >> > That is not really the issue, is it? The real issue is that for some > reason shorewall is attempting access to "/etc" (and, by extension, > "/etrc/shorewall") without me asking (or granting) shorewall such access. > >> /sbin/shorewall reads shorewall.conf prior to launching the compiler. >> > So? Again, that is not the issue - the issue is ... see above, I can''t > be bothered. > >> Given that shorewall.conf can use variables from params, the params file >> is read first. > And? > >> Because shorewall.conf has not be read yet, its >> CONFIG_PATH setting is not yet available. >> >> The CONFIG_PATH used is formed by prepending the directory named in the >> compile or check command, to the default CONFIG_PATH. >> >> So, at a minimum, the directory named in a ''compile'' >> or ''check'' command must contain a params file and shorewall.conf. >> > I''ve never said/nor implied that the presence/absence of "params" (or > any other) file is the issue. It isn''t! For the last time - why is > shorewall reading/attempting to access my "/etc" directory (and, by > extension, "/etc/shorewall"), given that I have not specified, nor > desire, nor granted shorewall such access? > > Second question: why is shorewall modifying my CONFIG_PATH to include > "/etc" (and, by extension, "/etc/shorewall") - I''ve never specified such > directory, nor desired (or granted) shorewall such access when executing > "shorewall compile . firewall" to produce a > separate-and-nothing-to-do-with-my-host-shorewall-configuration firewall > script? > > If you are still struggling to comprehend the above points, here is the > short version - I have no problems with shorewall screaming at me > if/when "params" (or anything else) is missing - what I have an issue > with is that shorewall, for whatever reason, unilaterally decided to > access "/etc" (and, by extension, "/etc/shorewall") and place that > directory in the middle of my CONFIG_PATH variable, without my knowledge > or consent, screwing everything up in the process. > > The access to /"etc" was restricted by myself, deliberately, so that I > could catch shorewall with its pants down (as evident from the above) if > it attempts such access when none is desired, when I produce my > shorewall firewall script designed to be used elsewhere.See the first ''New Feature'' in 4.5.14 Beta 1 -Tom PS -- hope you don''t struggle to comprehend why that change addresses your problem. -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb