I have just uploaded 4.3.9.
I''m particularly insterested in people testing the last new feature
(IPMARK and ''occurs'') and helping me update the Traffic
Shaping
documentation with a realistic example of its use.
----------------------------------------------------------------------------
P R O B L E M S C O R R E C T E D I N 4 . 3 . 9
----------------------------------------------------------------------------
1) If an interface name was specified in a PREROUTING rule (tcrules or
notrack) then an invalid iptables rule would be generated.
2) When a network address appeared in the SOURCE or DEST column of
/etc/shorewall/tcfilters, an incorrect netmask was being generated in
the resulting ''tc'' rule.
3) Placing an IP address in the BROADCAST column of
/etc/shorewall/interfaces caused Shorewall-perl to generate an
erroneous ''Invalid BROADCAST address'' error.
----------------------------------------------------------------------------
K N O W N P R O B L E M S R E M A I N I N G
----------------------------------------------------------------------------
None.
----------------------------------------------------------------------------
N E W F E A T U R E S I N 4 . 3 . 9
----------------------------------------------------------------------------
1) Rules that specify a log level with a target other than LOG or NFLOG
are now implemented through a separate chain. While this may
increase the processing cost slightly for packets that match these
rules, it is expected to reduce the overall cost of such rules
because each packet that doesn''t match the rules only has to be
processed once per rule rather than twice.
Example:
/etc/shorewall/rules:
REJECT:info loc net tcp 25
This previously generated these two rules (long rules folded):
-A loc2net -p 6 --dport 25 -j LOG --log-level 6
--log-prefix "Shorewall:loc2net:reject:"
-A loc2net -p 6 --dport 25 -j reject
It now generates these rules:
:log0 - [0:0]
...
-A loc2net -p 6 --dport 25 -g log0
...
-A log0 -j LOG --log-level 6
--log-prefix "Shorewall:loc2net:REJECT:"
-A log0 -p 6 --dport 25 -j reject
Notice that now there is only a single rule generated in the
''loc2net'' chain where before there were two. Packets for
other than
TCP port 25 had to be processed by both rules.
Notice also that the new LOG rule reflects the original action
("REJECT") rather than what Shorewall maps that to
("reject").
2) Shorewall6 has now been tested on kernel 2.6.24 (Ubuntu Hardy) and
hence will now start successfully when running on that kernel.
3) Three new options (IP, TC and IPSET) have been added to
shorewall.conf and shorwall6.conf. These options specify the name
of the executable for the ''ip'', ''tc'' and
''ipset'' utilities
respectively.
If not specified, the default values are:
IP=ip
TC=tc
IPSET=ipset
In other words, the utilities will be located via the current PATH
setting.
4) There has been a desire in the user community to limit traffic by
IP address using Shorewall traffic shaping. Heretofore, that has
required a very inefficient process:
a) Define a tcclass for each internal host (two, if shaping both in
and out).
b) Define a tcrule for each host to mark to classify the packets
accordingly.
Beginning with Shorewall 4.3.9, this process is made easier IF YOU
ARE WILLING TO INSTALL xtables-addons. The feature requires IPMARK
support in iptables[6] and your kernel. That support is available
in xtables-addons.
The new facility has two components:
a) A new IPMARK MARKing command in /etc/shorewall/tcrules.
b) A new ''occurs'' OPTION in /etc/shorewall/tcclasses.
The IPMARK target assigns a mark to each matching packet based on
the either the source or destination IP address. By default, it
assigns a mark value equal to the low-order 8 bits of the source
address.
The syntax is as follows:
IPMARK[([{src|dst}][,[<mask1>][,[<mask2>][,[<shift>]]]])]
Default values are:
src
<mask1> = 0xFF
<mask2> = 0x00
<shift> = 0
''src'' and ''dst'' specify whether the
mark is to be based on the
source or destination address respectively.
The selected address is first LANDed with <mask1> then LORed with
<mask2>.
The result is then shifted <shift> bits to the right.
Example:
IPMARK(dst, 0XFF00, 0x8000,8)
Destination IP address is 192.168.4.3 = 0xc0a80103
0xc0a80403 LAND 0xFF00 = 0x0400
0x0400 LOR 0x80 = 0x8400
0x8400 >> 8 = 0x84
Mark = 0x84 = 132
The ''occurs'' option causes the class definition to be
replicated
many times. The synax is:
occurs=<number>
When ''occurs'' is used:
a) The associated device may not have the ''classify''
option.
b) The class may not be the default class.
c) The class may not have any ''tos='' options
(including
''tcp-ack'').
The ''RATE'' and ''CEIL'' parameters apply
to each instance of the
class. So the total RATE represented by an entry with
''occurs'' will
be the listed RATE multiplied by the ''occurs'' number.
Example:
#DEVICE MARK RATE CEIL PRIORITY OPTIONS
eth0 100 1kbit 230kbit 4 occurs=32
The above defines 32 classes with MARK values 100-131. Each
class has a guaranteed rate of 1kbit/second.
As part of this change, the generation of class ids from mark
values has been changed. The class number is now
( <devnum> << 10 ) | <mask>
/sbin/shorewall has an ''encode'' and
''decode'' command to translate a
device number, mark pair to/from a classid:
encode <devnum> <mark>
decode <classnum>
Example:
$ shorewall decode 3172
Device = 3 Mark = 100
$ shorewall encode 3 100
Class number = 3172
$
-Tom
--
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
Tom I have been testing IPMARK in environment: kernel 2.6.29 iptables 1.4.3.2 xtables-addons 1.13 Iptables allows mask1 and mask2 to be in the range -4294967295 to 4294967295 or -0XFFFFFFFF to 0XFFFFFFFF. If a value is entered that is outside this range, Shorewall allows it but iptables-restore produces the following message: iptables-restore v1.4.3.2: IPMARK: Bad value for "and-mask" option: "-4294967296" Additionally if the value is in hex and is greater than 0XFFFFFFFF the following shorewall messages are produced: Integer overflow in hexadecimal number at /usr/share/shorewall/Shorewall/Config.pm line 738, <$currentfile> line 45. Hexadecimal number > 0xffffffff non-portable at /usr/share/shorewall/Shorewall/Config.pm line 738, <$currentfile> line 45. Similarly iptables allows the value of shift to be in the range 0 to 128 or 0 to 0x80. If a value outside this range is entered, shorewall allows it but iptables-restore produces the following message: iptables-restore v1.4.3.2: IPMARK: Bad value for "--shift" option: "154" ------------------------------------------------------------------------------------------------------------ Issuing ''shorewall6 show -f capabilities'' gives: CAPVERSION=40205 Creating a capabilities file, then issuing "shorewall6 check" produces the following message: WARNING: Your capabilities file is out of date -- it does not contain all of the capabilities defined by Shorewall6 version 4.3.9 Manually changing CAPVERSION to 40309 in the capabilities file stops the message being produced. Using IPMARK in /etc/shorewall6/tcrules produces the following message: ERROR: IPMARK requires IPMARK Target in your kernel and iptables : /etc/shorewall6/tcrules (line 22) I don''t know if this a Shorewall6 issue or there is a problem with my setup. ------------------------------------------------------------------------------------------------------------------ There is a minor typo in the release notes on the following line: Destination IP address is 192.168.4.3 = 0xc0a80103 The IP address doesn''t tie up with the hex value. Steven. ------------------------------------------------------------------------------ Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p
On Mon, Apr 20, 2009 at 08:56:06PM +0100, Steven Jan Springl wrote:> I have been testing IPMARK in environment: > > kernel 2.6.29 > iptables 1.4.3.2 > xtables-addons 1.13 > > Iptables allows mask1 and mask2 to be in the range -4294967295 to 4294967295 > or -0XFFFFFFFF to 0XFFFFFFFF. If a value is entered that is outside this > range, Shorewall allows it but iptables-restore produces the following > message:A 33 bit value? Neat. -- Len Sorensen ------------------------------------------------------------------------------ Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p
Steven Jan Springl wrote:> Tom > > I have been testing IPMARK in environment: > > kernel 2.6.29 > iptables 1.4.3.2 > xtables-addons 1.13 > > Iptables allows mask1 and mask2 to be in the range -4294967295 to 4294967295 > or -0XFFFFFFFF to 0XFFFFFFFF. If a value is entered that is outside this > range, Shorewall allows it but iptables-restore produces the following > message: > > iptables-restore v1.4.3.2: IPMARK: Bad value for "and-mask" > option: "-4294967296" > > Additionally if the value is in hex and is greater than 0XFFFFFFFF the > following shorewall messages are produced: > > Integer overflow in hexadecimal number > at /usr/share/shorewall/Shorewall/Config.pm line 738, <$currentfile> line 45. > Hexadecimal number > 0xffffffff non-portable > at /usr/share/shorewall/Shorewall/Config.pm line 738, <$currentfile> line 45. > > Similarly iptables allows the value of shift to be in the range 0 to 128 or 0 > to 0x80. If a value outside this range is entered, shorewall allows it but > iptables-restore produces the following message: > > iptables-restore v1.4.3.2: IPMARK: Bad value for "--shift" option: "154" > > ------------------------------------------------------------------------------------------------------------ > > Issuing ''shorewall6 show -f capabilities'' gives: > > CAPVERSION=40205 > > Creating a capabilities file, then issuing "shorewall6 check" produces the > following message: > > WARNING: Your capabilities file is out of date -- it does not contain all of > the capabilities defined by Shorewall6 version 4.3.9 > > Manually changing CAPVERSION to 40309 in the capabilities file stops the > message being produced. > > Using IPMARK in /etc/shorewall6/tcrules produces the following message: > > ERROR: IPMARK requires IPMARK Target in your kernel and > iptables : /etc/shorewall6/tcrules (line 22) > > I don''t know if this a Shorewall6 issue or there is a problem with my setup. > > ------------------------------------------------------------------------------------------------------------------ > > There is a minor typo in the release notes on the following line: > > Destination IP address is 192.168.4.3 = 0xc0a80103 > > The IP address doesn''t tie up with the hex value.Thanks, Steven. I believe that all issues are corrected in Git commit 34791612b537b90ceb76edf31fce2f299e687bee. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p
Tom Eastep wrote:> > Thanks, Steven. > > I believe that all issues are corrected in Git commit > 34791612b537b90ceb76edf31fce2f299e687bee. >Oops -- another fix in bc3424995b418fb4ea4e032065a91662d85bd46b. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p
On Monday 20 April 2009 21:37:36 Tom Eastep wrote:> Tom Eastep wrote: > > Thanks, Steven. > > > > I believe that all issues are corrected in Git commit > > 34791612b537b90ceb76edf31fce2f299e687bee. > > Oops -- another fix in bc3424995b418fb4ea4e032065a91662d85bd46b. > > -TomTom After the applying the above fixes, specifying a negative hex value (-0x8800) in mask1 produces the following message: Argument "-0x8800" isn''t numeric in numeric le (<=) at /usr/share/shorewall/Shorewall/Tc.pm line 319, <$currentfile> line 46. A negative hex value (-0xcc00) in mask2 produces the following message: Argument "-0xcc00" isn''t numeric in numeric le (<=) at /usr/share/shorewall/Shorewall/Tc.pm line 325, <$currentfile> line 46. ------------------------------------------------------------------------------------------------------------------ Prior to the application of the above patches, specifying a mark value of 1 in the providers file was allowed. After the application of the patches, a mark value of 1 is no longer allowed. I have tried setting all combinations of Yes and No in WIDE_TC_MARKS and HIGH_ROUTE_MARKS. Is this expected? Steven. ------------------------------------------------------------------------------ Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p
Tom Eastep wrote:> Steven Jan Springl wrote: >> Prior to the application of the above patches, specifying a mark value of 1 in >> the providers file was allowed. >> After the application of the patches, a mark value of 1 is no longer allowed. >> I have tried setting all combinations of Yes and No in WIDE_TC_MARKS and >> HIGH_ROUTE_MARKS. >> Is this expected? >> > > No, it is not. I''ll try to have a look this evening.This should be corrected by dcee6562a29ce4f7aac353cf1cb87c2814c7522a. Thanks Steven, -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p
Steven Jan Springl wrote:> > After the applying the above fixes, specifying a negative hex value (-0x8800) > in mask1 produces the following message: > > Argument "-0x8800" isn''t numeric in numeric le (<=) > at /usr/share/shorewall/Shorewall/Tc.pm line 319, <$currentfile> line 46. > > A negative hex value (-0xcc00) in mask2 produces the following message: > > Argument "-0xcc00" isn''t numeric in numeric le (<=) > at /usr/share/shorewall/Shorewall/Tc.pm line 325, <$currentfile> line 46.I have fixed this particular case but I''m guessing that this same thing can be produced at will all over the configuration files. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p