LARTC - Dec 2006 - iptables 1.3.7, kernel 2.6.19, ROUTE and Layer7 issues

Hi, I''m having problems with this configuration:
   iptables 1.3.7 (vanilla or repackaged for fc5)
   kernel 2.6.19 (vanilla)
   ROUTE 1.11 (last pom-ng)
   layer7-filter 2.6 (last in sf.net)
   connlimit (last pom-ng)

When I try to use -j ROUTE in any chain in mangle table I have this error:

[root@myhost ~]# iptables -v -t mangle -A POSTROUTING -p tcp --dport msnp
-j ROUTE --gw $chat_gw
ROUTE  tcp opt -- in * out *  0.0.0.0/0  -> 0.0.0.0/0  tcp dpt:1863 ROUTE
gw:80.32.61.1
iptables: Invalid argument

[root@myhost ~]# dmesg | grep "ROUTE"
ipt_ROUTE: targinfosize 0 != 40

[root@myhost ~]# cat /var/log/messages | grep "ROUTE"
Dec 11 20:32:50 myhost kernel: ipt_ROUTE: targinfosize 0 != 40

With layer7 filter, I have a problem too, but it has no dmesg or syslog
entry:

[root@myhost ~]# iptables -v -t mangle -A PREROUTING -m layer7 --l7proto
msnmessenger
  0 opt -- in * out *  0.0.0.0/0  -> 0.0.0.0/0  LAYER7 l7proto msnmessenger
iptables: Invalid argument

Does anyone help me please? I need any more recent patch?

More info:
   SMP machine (dual Xeon)

Thanks

Any help about this?

Thanks

El Lun, 11 de Diciembre de 2006, 20:44, ArcosCom Linux User
escribió:
> Hi, I''m having problems with this configuration:
>    iptables 1.3.7 (vanilla or repackaged for fc5)
>    kernel 2.6.19 (vanilla)
>    ROUTE 1.11 (last pom-ng)
>    layer7-filter 2.6 (last in sf.net)
>    connlimit (last pom-ng)
>
> When I try to use -j ROUTE in any chain in mangle table I have this error:
>
> [root@myhost ~]# iptables -v -t mangle -A POSTROUTING -p tcp --dport msnp
> -j ROUTE --gw $chat_gw
> ROUTE  tcp opt -- in * out *  0.0.0.0/0  -> 0.0.0.0/0  tcp dpt:1863
ROUTE
> gw:80.32.61.1
> iptables: Invalid argument
>
> [root@myhost ~]# dmesg | grep "ROUTE"
> ipt_ROUTE: targinfosize 0 != 40
>
> [root@myhost ~]# cat /var/log/messages | grep "ROUTE"
> Dec 11 20:32:50 myhost kernel: ipt_ROUTE: targinfosize 0 != 40
>
> With layer7 filter, I have a problem too, but it has no dmesg or syslog
> entry:
>
> [root@myhost ~]# iptables -v -t mangle -A PREROUTING -m layer7 --l7proto
> msnmessenger
>   0 opt -- in * out *  0.0.0.0/0  -> 0.0.0.0/0  LAYER7 l7proto
> msnmessenger
> iptables: Invalid argument
>
> Does anyone help me please? I need any more recent patch?
>
> More info:
>    SMP machine (dual Xeon)
>
> Thanks
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>

ArcosCom Linux User wrote:
> El Lun, 11 de Diciembre de 2006, 20:44, ArcosCom Linux User escribió:
> 
>>Hi, I''m having problems with this configuration:
>>   iptables 1.3.7 (vanilla or repackaged for fc5)
>>   kernel 2.6.19 (vanilla)
>>   ROUTE 1.11 (last pom-ng)
>>   layer7-filter 2.6 (last in sf.net)
>>   connlimit (last pom-ng)
>>
>>When I try to use -j ROUTE in any chain in mangle table I have this
error:
>>
>>[root@myhost ~]# iptables -v -t mangle -A POSTROUTING -p tcp --dport
msnp
>>-j ROUTE --gw $chat_gw
>>ROUTE  tcp opt -- in * out *  0.0.0.0/0  -> 0.0.0.0/0  tcp dpt:1863
ROUTE
>>gw:80.32.61.1
>>iptables: Invalid argument
>>
>>[root@myhost ~]# dmesg | grep "ROUTE"
>>ipt_ROUTE: targinfosize 0 != 40

The ROUTE target needs to set the targetsize field in struct ipt_target.
It probably needs other adjustments for 2.6.19 as well. I would just use
normal policy routing ..

Thanks for your response.

I''m using multiple gateways for internet connection and having problems
with random disconection, and I not use ROUTE usually, but I was trying to
force only one gateway for one type of traffic (which the clients lost
conections and are having issues).

I know I can use -j MARK or -j CONNMARK and this mark to filter, but
I''m
using marks for another purposes and I can''t use it for routing.

The box is a dual xeon and the kernel has been compiled SMP enabled.

I haven''t tested ROUTE yet with this kernel (2.6.19), but with 2.6.18.x
I
were having a problem with -j ROUTE in -t mangle and POSTROUTING chain.

Perhaps ROUTE need a more in deepth revision?

Do I help more reporting the bug into netfilter-bugzilla?

Thanks a lot.

El Mar, 12 de Diciembre de 2006, 9:34, Patrick McHardy
escribió:
> ArcosCom Linux User wrote:
>> El Lun, 11 de Diciembre de 2006, 20:44, ArcosCom Linux User escribió:
>>
>>>Hi, I''m having problems with this configuration:
>>>   iptables 1.3.7 (vanilla or repackaged for fc5)
>>>   kernel 2.6.19 (vanilla)
>>>   ROUTE 1.11 (last pom-ng)
>>>   layer7-filter 2.6 (last in sf.net)
>>>   connlimit (last pom-ng)
>>>
>>>When I try to use -j ROUTE in any chain in mangle table I have this
>>> error:
>>>
>>>[root@myhost ~]# iptables -v -t mangle -A POSTROUTING -p tcp --dport
>>> msnp
>>>-j ROUTE --gw $chat_gw
>>>ROUTE  tcp opt -- in * out *  0.0.0.0/0  -> 0.0.0.0/0  tcp
dpt:1863
>>> ROUTE
>>>gw:80.32.61.1
>>>iptables: Invalid argument
>>>
>>>[root@myhost ~]# dmesg | grep "ROUTE"
>>>ipt_ROUTE: targinfosize 0 != 40
>
>
> The ROUTE target needs to set the targetsize field in struct ipt_target.
> It probably needs other adjustments for 2.6.19 as well. I would just use
> normal policy routing ..
>
>
>

ArcosCom Linux User wrote:
> Thanks for your response.
> 
> I''m using multiple gateways for internet connection and having
problems
> with random disconection, and I not use ROUTE usually, but I was trying to
> force only one gateway for one type of traffic (which the clients lost
> conections and are having issues).
> 
> I know I can use -j MARK or -j CONNMARK and this mark to filter, but
I''m
> using marks for another purposes and I can''t use it for routing.
Everything using marks supports bitmasks in 2.6.19.
> The box is a dual xeon and the kernel has been compiled SMP enabled.
> 
> I haven''t tested ROUTE yet with this kernel (2.6.19), but with
2.6.18.x I
> were having a problem with -j ROUTE in -t mangle and POSTROUTING chain.
> 
> Perhaps ROUTE need a more in deepth revision?
As I said, it needs to fill in the targetsize field and probably needs
to adjust the target function signature.
> Do I help more reporting the bug into netfilter-bugzilla?
Its still down, but the ROUTE patch is unmaintained anyway.

Then, the actual and updated and maintained substitute for ROUTE is using
CONNMARK and/or MARK and then add filters/rules to routes table with ip.
Am I in the truth?

Sorry for my out-of-date knoledge of these things and for the
"obvious"
questions.

Thanks a lot.

El Mie, 13 de Diciembre de 2006, 9:38, Patrick McHardy
escribió:
> ArcosCom Linux User wrote:
>> Thanks for your response.
>>
>> I''m using multiple gateways for internet connection and having
problems
>> with random disconection, and I not use ROUTE usually, but I was trying
>> to
>> force only one gateway for one type of traffic (which the clients lost
>> conections and are having issues).
>>
>> I know I can use -j MARK or -j CONNMARK and this mark to filter, but
I''m
>> using marks for another purposes and I can''t use it for
routing.
>
> Everything using marks supports bitmasks in 2.6.19.
>
>> The box is a dual xeon and the kernel has been compiled SMP enabled.
>>
>> I haven''t tested ROUTE yet with this kernel (2.6.19), but with
2.6.18.x
>> I
>> were having a problem with -j ROUTE in -t mangle and POSTROUTING chain.
>>
>> Perhaps ROUTE need a more in deepth revision?
>
> As I said, it needs to fill in the targetsize field and probably needs
> to adjust the target function signature.
>
>> Do I help more reporting the bug into netfilter-bugzilla?
>
> Its still down, but the ROUTE patch is unmaintained anyway.
>

ArcosCom Linux User wrote:
> Then, the actual and updated and maintained substitute for ROUTE is using
> CONNMARK and/or MARK and then add filters/rules to routes table with ip.
> Am I in the truth?

That has always been the better way. The route target is a hack, I''m
don''t know why it exists at all.

>> Do I help more reporting the bug into netfilter-bugzilla?
>
>Its still down, but the ROUTE patch is unmaintained anyway.
I take it...! ;)

(In fact I think I fixed that targetsize thing a whee while ago in my own
iptables and kernel packages.)

	-`J''
--

>> Then, the actual and updated and maintained substitute for ROUTE is
using
>> CONNMARK and/or MARK and then add filters/rules to routes table with
ip.
>> Am I in the truth?
>
>That has always been the better way. The route target is a hack,
I''m
>don''t know why it exists at all.
The biggest problem with Linux is that you have to match packets thrice 
(three times) if you use all of the fluffy features (netfilter, policy 
routing and QoS). That''s a real PITA and a CPU burner too.

Well, as far as an example goes it could be like

  iptables -m time --blah sundays -j ROUTE -o eth1 (terminating target)
  default route eth0

On the other hand, using MARK would require

  iptables -t mangle -A OUTPUT -m time --sundays -j MARK --set-mark 0x1
  ip route ... funky comand matching 0x1

Plus there is yet some repition inside iptables itself, e.g. if one was 
able to MARK packets while they were filtered:

Requirement as of today: MARK must be in mangle. If you have tons of 
rules, you need a lot of MARK operations. However, if MARK could be used 
inside the filter table, you could drop all packets you are not 
interested in first, then mark them.


	-`J''
--

On Wed, 13 Dec 2006, Patrick McHardy wrote:
> ArcosCom Linux User wrote:
>> Thanks for your response.
>>
>> I''m using multiple gateways for internet connection and having
problems
>> with random disconection, and I not use ROUTE usually, but I was trying
to
>> force only one gateway for one type of traffic (which the clients lost
>> conections and are having issues).
>>
>> I know I can use -j MARK or -j CONNMARK and this mark to filter, but
I''m
>> using marks for another purposes and I can''t use it for
routing.
>
> Everything using marks supports bitmasks in 2.6.19.
>
>> The box is a dual xeon and the kernel has been compiled SMP enabled.
>>
>> I haven''t tested ROUTE yet with this kernel (2.6.19), but with
2.6.18.x I
>> were having a problem with -j ROUTE in -t mangle and POSTROUTING chain.
>>
>> Perhaps ROUTE need a more in deepth revision?
>
> As I said, it needs to fill in the targetsize field and probably needs
> to adjust the target function signature.
>
>> Do I help more reporting the bug into netfilter-bugzilla?
>
> Its still down, but the ROUTE patch is unmaintained anyway.
How about attached (and inlined) patch. BTW - is it possible to add a 
Kconfig entry after a specific text, like with Makefile.ladd?


[POM-NG] ROUTE: 2.6.19 compatibility fix

Make both IPv4 and IPv6 versions compatible with 2.6.19

Signed-off-by: Krzysztof Piotr Oledzki <ole@ans.pl>

diff -Nur
patch-o-matic-ng-20061221-orig/patchlets/ROUTE/linux-2.6/net/ipv4/netfilter/ipt_ROUTE.c
patch-o-matic-ng-20061221/patchlets/ROUTE/linux-2.6/net/ipv4/netfilter/ipt_ROUTE.c
---
patch-o-matic-ng-20061221-orig/patchlets/ROUTE/linux-2.6/net/ipv4/netfilter/ipt_ROUTE.c
2006-07-22 16:07:52.000000000 +0200
+++
patch-o-matic-ng-20061221/patchlets/ROUTE/linux-2.6/net/ipv4/netfilter/ipt_ROUTE.c
2006-12-28 21:13:07.000000000 +0100
@@ -17,6 +17,7 @@
  #include <linux/netfilter_ipv4/ipt_ROUTE.h>
  #include <linux/netdevice.h>
  #include <linux/route.h>
+#include <linux/version.h>
  #include <linux/if_arp.h>
  #include <net/ip.h>
  #include <net/route.h>
@@ -280,9 +281,15 @@
  				     const struct net_device *in,
  				     const struct net_device *out,
  				     unsigned int hooknum,
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,17)
  				     const struct xt_target *target,
+#endif
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19)
  				     const void *targinfo,
  				     void *userinfo)
+#else
+				     const void *targinfo)
+#endif
  {
  	const struct ipt_route_target_info *route_info = targinfo;
  	struct sk_buff *skb = *pskb;
@@ -401,10 +408,18 @@


  static int ipt_route_checkentry(const char *tablename,
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,16)
  				const void *e,
+#else
+				const struct ipt_ip *ip,
+#endif
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,17)
  				const struct xt_target *target,
+#endif
  				void *targinfo,
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19)
  				unsigned int targinfosize,
+#endif
  				unsigned int hook_mask)
  {
  	if (strcmp(tablename, "mangle") != 0) {
@@ -422,12 +437,14 @@
  		return 0;
  	}

+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19)
  	if (targinfosize != IPT_ALIGN(sizeof(struct ipt_route_target_info))) {
  		printk(KERN_WARNING "ipt_ROUTE: targinfosize %u != %Zu\n",
  		       targinfosize,
  		       IPT_ALIGN(sizeof(struct ipt_route_target_info)));
  		return 0;
  	}
+#endif

  	return 1;
  }
@@ -436,7 +453,9 @@
  static struct ipt_target ipt_route_reg = {
  	.name = "ROUTE",
  	.target = ipt_route_target,
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,17)
  	.targetsize = sizeof(struct ipt_route_target_info),
+#endif
  	.checkentry = ipt_route_checkentry,
  	.me = THIS_MODULE,
  };
diff -Nur
patch-o-matic-ng-20061221-orig/patchlets/ROUTE/linux-2.6/net/ipv6/netfilter/Kconfig.ladd
patch-o-matic-ng-20061221/patchlets/ROUTE/linux-2.6/net/ipv6/netfilter/Kconfig.ladd
---
patch-o-matic-ng-20061221-orig/patchlets/ROUTE/linux-2.6/net/ipv6/netfilter/Kconfig.ladd
2004-05-06 14:41:19.000000000 +0200
+++
patch-o-matic-ng-20061221/patchlets/ROUTE/linux-2.6/net/ipv6/netfilter/Kconfig.ladd
2006-12-28 19:07:03.000000000 +0100
@@ -1,5 +1,5 @@
  config IP6_NF_TARGET_ROUTE
-	tristate ''    ROUTE target support''
+	tristate ''ROUTE target support''
  	depends on IP6_NF_MANGLE
  	help
  	  This option adds a `ROUTE'' target, which enables you to setup
unusual
diff -Nur
patch-o-matic-ng-20061221-orig/patchlets/ROUTE/linux-2.6/net/ipv6/netfilter/ip6t_ROUTE.c
patch-o-matic-ng-20061221/patchlets/ROUTE/linux-2.6/net/ipv6/netfilter/ip6t_ROUTE.c
---
patch-o-matic-ng-20061221-orig/patchlets/ROUTE/linux-2.6/net/ipv6/netfilter/ip6t_ROUTE.c
2006-08-08 11:56:40.000000000 +0200
+++
patch-o-matic-ng-20061221/patchlets/ROUTE/linux-2.6/net/ipv6/netfilter/ip6t_ROUTE.c
2006-12-28 21:13:16.000000000 +0100
@@ -15,6 +15,7 @@
  #include <linux/netfilter_ipv6/ip6_tables.h>
  #include <linux/netfilter_ipv6/ip6t_ROUTE.h>
  #include <linux/netdevice.h>
+#include <linux/version.h>
  #include <net/ipv6.h>
  #include <net/ndisc.h>
  #include <net/ip6_route.h>
@@ -192,9 +193,15 @@
  		  const struct net_device *in,
  		  const struct net_device *out,
  		  unsigned int hooknum,
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,17)
  		  const struct xt_target *target,
+#endif
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19)
  		  const void *targinfo,
  		  void *userinfo)
+#else
+		  const void *targinfo)
+#endif
  {
  	const struct ip6t_route_target_info *route_info = targinfo;
  	struct sk_buff *skb = *pskb;
@@ -260,10 +267,18 @@

  static int
  ip6t_route_checkentry(const char *tablename,
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,16)
  		      const void *entry,
+#else
+		      const struct ip6t_entry *entry
+#endif
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,17)
  		      const struct xt_target *target,
+#endif
  		      void *targinfo,
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19)
  		      unsigned int targinfosize,
+#endif
  		      unsigned int hook_mask)
  {
  	if (strcmp(tablename, "mangle") != 0) {
@@ -271,12 +286,14 @@
  		return 0;
  	}

+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19)
  	if (targinfosize != IP6T_ALIGN(sizeof(struct ip6t_route_target_info))) {
  		printk(KERN_WARNING "ip6t_ROUTE: targinfosize %u != %Zu\n",
  		       targinfosize,
  		       IP6T_ALIGN(sizeof(struct ip6t_route_target_info)));
  		return 0;
  	}
+#endif

  	return 1;
  }
@@ -285,7 +302,9 @@
  static struct ip6t_target ip6t_route_reg = {
  	.name       = "ROUTE",
  	.target     = ip6t_route_target,
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,17)
  	.targetsize = sizeof(struct ip6t_route_target_info),
+#endif
  	.checkentry = ip6t_route_checkentry,
  	.me         = THIS_MODULE
  };

Best regards,

 				Krzysztof Olędzki

Krzysztof Oledzki wrote:
>> Its still down, but the ROUTE patch is unmaintained anyway.
> 
> 
> How about attached (and inlined) patch. BTW - is it possible to add a
> Kconfig entry after a specific text, like with Makefile.ladd?
> 
> 
> [POM-NG] ROUTE: 2.6.19 compatibility fix
> 
> Make both IPv4 and IPv6 versions compatible with 2.6.19

Thanks Krzysztof, applied.

I would prefer to have someone maintain it externally though. Jan, are
you still interested in doing that? If you need help or webspace for
an external repository please let me know.

On Jan 10 2007 06:58, Patrick McHardy wrote:
>Krzysztof Oledzki wrote:
>>> Its still down, but the ROUTE patch is unmaintained anyway.
>> 
>> How about attached (and inlined) patch. BTW - is it possible to add a
>> Kconfig entry after a specific text, like with Makefile.ladd?
>> 
>> [POM-NG] ROUTE: 2.6.19 compatibility fix
>> 
>> Make both IPv4 and IPv6 versions compatible with 2.6.19
>
>
>Thanks Krzysztof, applied.
>
>I would prefer to have someone maintain it externally though. Jan, are
>you still interested in doing that? If you need help or webspace for
>an external repository please let me know.
I would give it a try. Though I would really prefer to have it in the
kernel and iptables rather than pomng or pomng-external. In my
opinion that simplifies maintainability. Changes in the netfilter API
seem to be the most common reason for patching (someone changed the
xt_match->match and xt_target->target signatures in 2.6.20 again!),
and keeping out-of-tree modules compiling with kernel-du-jour can be
an #ifdef pita. Then it''s really preferable to have 2.6.18 have a
xt_FOOBAR with netfilter-2.6.18 signatures, and 2.6.20 with
netfilter-2.6.20. Especially since many people run distributions with
RPM/DEBified iptables, so the POM `runme` will not be easy to
accomplish for the casual user. (I currently do have that issue -
after doing `svn up` on pomng, I have to manually move the changes to
(my) kernel rpm and (my) iptables rpm, because the days of `make
install` are GONE for me - at least I try.)

I understand that POM does not require to compile with all
kernels-of-the-last-three-months, but this also simplifies
integration for end users. They do not need to backport/forward port
indated/outdated out-of-tree modules and, at best, do not even need
to recompile the kernel.

Of course there are some modules that continue being out-of-tree
because they would not fit in (imagine a 500K geoip.c with a
compiled-in big string array). Not sure what to do about them.
Perhaps do it like chaostables [2.6.18-2.6.20], trying to keep it
working for a limited set of kernels.

Oh well, that said, my ideal plan would be to get ROUTE TARPIT
connlimit and u32 into mainline in one go, and perhaps, after review
and discussion, chaostables and some of the others that live in
Krzystof''s patchlet collection.

Opinions, please?


	-`J''
--

Jan Engelhardt wrote:
> On Jan 10 2007 06:58, Patrick McHardy wrote:
> 
>>I would prefer to have someone maintain it externally though. Jan, are
>>you still interested in doing that? If you need help or webspace for
>>an external repository please let me know.
> 
> 
> I would give it a try. Though I would really prefer to have it in the
> kernel and iptables rather than pomng or pomng-external. In my
> opinion that simplifies maintainability. Changes in the netfilter API
> seem to be the most common reason for patching (someone changed the
> xt_match->match and xt_target->target signatures in 2.6.20 again!),
> and keeping out-of-tree modules compiling with kernel-du-jour can be
> an #ifdef pita. Then it''s really preferable to have 2.6.18 have a
> xt_FOOBAR with netfilter-2.6.18 signatures, and 2.6.20 with
> netfilter-2.6.20. Especially since many people run distributions with
> RPM/DEBified iptables, so the POM `runme` will not be easy to
> accomplish for the casual user. (I currently do have that issue -
> after doing `svn up` on pomng, I have to manually move the changes to
> (my) kernel rpm and (my) iptables rpm, because the days of `make
> install` are GONE for me - at least I try.)
> 
> I understand that POM does not require to compile with all
> kernels-of-the-last-three-months, but this also simplifies
> integration for end users. They do not need to backport/forward port
> indated/outdated out-of-tree modules and, at best, do not even need
> to recompile the kernel.
> 
> Of course there are some modules that continue being out-of-tree
> because they would not fit in (imagine a 500K geoip.c with a
> compiled-in big string array). Not sure what to do about them.
> Perhaps do it like chaostables [2.6.18-2.6.20], trying to keep it
> working for a limited set of kernels.
> 
> Oh well, that said, my ideal plan would be to get ROUTE TARPIT
> connlimit and u32 into mainline in one go, and perhaps, after review
> and discussion, chaostables and some of the others that live in
> Krzystof''s patchlet collection.
ROUTE will not go in, its a bad hack and shouldn''t be used (which
is why I would prefer to get rid of it). Haven''t looked at TARPIT
and connlimit in a long time, we can think about it.

I think so, there are many old matches that are stables and I have to
apply many times when I update the kernel. If they where into kernel and
iptables (because they are now not as experimental than many months/years
ago) these problem when new kernel releases and/or iptables releases
disapears very quickly.

I have a great headache now, I had to patch my kernel, patch iptables,
update iproute to allow "mark-and" operations for routing. Yes, I can
adapt many thinks and forgot many routing/filtering functionality, but
then, my linux box will be useless for the purposes I deploy it.

I have no problem in patch and upgrade thinks, my problem is that I have
no time to do all these steps every any important bug, improvement is
released.

To allow compatibility with my installed distro (one in production and
another in testing), I had to readapt the RPM''s and modify specs to put
there the patch. Usually is easy, but when netfilter changes its internals
structs (that I have no knoledgement) I can''t correct the patches and
adapt them.

When I readapt RPM''s, I can use them into my systems, and I think many
users should want them for their systems and the same purpose as me and I
don''t know where I can send them to allow people to use them (perhaps I
make a simple webpage for that and a RPM repository, but I can''t use my
lines to allow comunity to massive download the rpm''s).

Too punctual work that comunity can''t improve or maintain because they
hasn''t got access. Perhaps, when I finish the testing and all the
problems
I have got in testing environtment were solved, I can make a good
repository to allow community to download and upload thinks, but now, I
had time to make it.

The real problems of all these, I think, are :
   1) That the time from a fix/release goes out to that change appears
into the distro repositories are enought.
   2) Another problem is when there are bugs are fixed in versios and that
fix is not backported to prior versions and distros mainteneers have to
backport the fix to preserve stable state of the distro package (less
features than actual stable sources, but stable system).
   3) Another problem is that the patches for bugfixes are not public or
easy accesible for normal users as me, and then I not only need to
learn how to patch, make and upgrade (or rpm/deb build), I need to
learn too how to extract the bugfix patch I need to solve one problem
in my kernel/iptables/module/etc... version, what are the developer
list, subscribe, where are the development sources that correct the
problem, how I can download the patch, etc...

I know that all these project are diferent communities, but if all
comunities make more easy to the end user some tasks, the users learns
more quickly how to contribute and work with the many projects and
tecknologies.

Don''t know how projects could make the thinks easy, but, in case of
netfilter, some tables explaining what matches are included in kernel (and
from what version) allow users to know what kernel version can use for
certain purposes.

For example, I know h323 where added into kernel at 2.6.17 kernel version,
but the community appears to be working into 2.6.16.x version to stabilize
it and and backported many bugfixes from 2.6.19.x to 2.6.16.x. I think
2.6.16.x kernel is the more stable at these days, but if I want h323
conntrack modules I can''t use pom-ńg because h323 is not in
pom-ng
today, and if I use old pom-ng, I know I will use a module with some bugs
corrected in 2.6.18.x kernel. Do I explained a problem that have many
users than me?

Another example, some body say in the lists that using "mark" will
allow
me to change routes by "mark value" or by "and-mark mask",
fine, that were
to substitute ROUTE tarjet. I think that is a correct solution but ...
What are the implications? 1) I can''t see any documentation in
netfilter
or iproute.
2) My distro don''t update the packages, I have to do it manually.
3) When I have an updated iproute package, kernel package, iptables
package and I finish and test the configuration, I have the knowledte to
write a little explanation/how-to to allow users to know how and don''t
waste their time looking for the steps ... some hours after I found a
wiki, I need to register, etc.... When I finished all the needed steps to
allow me to write a "more or less" public super-mini-howto I forgot
the
steps and I, finally, don''t write the info, because my systems are
working
for my purposes, I have my paper with my hand annotations and I need to
continue working in more tasks.

Sorry for the big e-mail, but it is only my user opinion about these type
o things.

Sorry for my poor english.

Regards

El Mie, 10 de Enero de 2007, 12:53, Jan Engelhardt
escribió:
>
> On Jan 10 2007 06:58, Patrick McHardy wrote:
>>Krzysztof Oledzki wrote:
>>>> Its still down, but the ROUTE patch is unmaintained anyway.
>>>
>>> How about attached (and inlined) patch. BTW - is it possible to add
a
>>> Kconfig entry after a specific text, like with Makefile.ladd?
>>>
>>> [POM-NG] ROUTE: 2.6.19 compatibility fix
>>>
>>> Make both IPv4 and IPv6 versions compatible with 2.6.19
>>
>>
>>Thanks Krzysztof, applied.
>>
>>I would prefer to have someone maintain it externally though. Jan, are
>>you still interested in doing that? If you need help or webspace for
>>an external repository please let me know.
>
> I would give it a try. Though I would really prefer to have it in the
> kernel and iptables rather than pomng or pomng-external. In my
> opinion that simplifies maintainability. Changes in the netfilter API
> seem to be the most common reason for patching (someone changed the
> xt_match->match and xt_target->target signatures in 2.6.20 again!),
> and keeping out-of-tree modules compiling with kernel-du-jour can be
> an #ifdef pita. Then it''s really preferable to have 2.6.18 have a
> xt_FOOBAR with netfilter-2.6.18 signatures, and 2.6.20 with
> netfilter-2.6.20. Especially since many people run distributions with
> RPM/DEBified iptables, so the POM `runme` will not be easy to
> accomplish for the casual user. (I currently do have that issue -
> after doing `svn up` on pomng, I have to manually move the changes to
> (my) kernel rpm and (my) iptables rpm, because the days of `make
> install` are GONE for me - at least I try.)
>
> I understand that POM does not require to compile with all
> kernels-of-the-last-three-months, but this also simplifies
> integration for end users. They do not need to backport/forward port
> indated/outdated out-of-tree modules and, at best, do not even need
> to recompile the kernel.
>
> Of course there are some modules that continue being out-of-tree
> because they would not fit in (imagine a 500K geoip.c with a
> compiled-in big string array). Not sure what to do about them.
> Perhaps do it like chaostables [2.6.18-2.6.20], trying to keep it
> working for a limited set of kernels.
>
> Oh well, that said, my ideal plan would be to get ROUTE TARPIT
> connlimit and u32 into mainline in one go, and perhaps, after review
> and discussion, chaostables and some of the others that live in
> Krzystof''s patchlet collection.
>
> Opinions, please?
>
>
> 	-`J''
> --
>
>

I think so, there are many old matches that are stables and I have to
apply many times when I update the kernel. If they where into kernel and
iptables (because they are now not as experimental than many months/years
ago) these problem when new kernel releases and/or iptables releases
disapears very quickly.

I have a great headache now, I had to patch my kernel, patch iptables,
update iproute to allow "mark-and" operations for routing. Yes, I can
adapt many thinks and forgot many routing/filtering functionality, but
then, my linux box will be useless for the purposes I deploy it.

I have no problem in patch and upgrade thinks, my problem is that I have
no time to do all these steps every any important bug, improvement is
released.

To allow compatibility with my installed distro (one in production and
another in testing), I had to readapt the RPM''s and modify specs to put
there the patch. Usually is easy, but when netfilter changes its internals
structs (that I have no knoledgement) I can''t correct the patches and
adapt them.

When I readapt RPM''s, I can use them into my systems, and I think many
users should want them for their systems and the same purpose as me and I
don''t know where I can send them to allow people to use them (perhaps I
make a simple webpage for that and a RPM repository, but I can''t use my
lines to allow comunity to massive download the rpm''s).

Too punctual work that comunity can''t improve or maintain because they
hasn''t got access. Perhaps, when I finish the testing and all the
problems
I have got in testing environtment were solved, I can make a good
repository to allow community to download and upload thinks, but now, I
had time to make it.

The real problems of all these, I think, are :
   1) That the time from a fix/release goes out to that change appears
into the distro repositories are enought.
   2) Another problem is when there are bugs are fixed in versios and that
fix is not backported to prior versions and distros mainteneers have to
backport the fix to preserve stable state of the distro package (less
features than actual stable sources, but stable system).
   3) Another problem is that the patches for bugfixes are not public or
easy accesible for normal users as me, and then I not only need to learn
how to patch, make and upgrade (or rpm/deb build), I need to learn too how
to extract the bugfix patch I need to solve one problem in my
kernel/iptables/module/etc... version, what are the developer list,
subscribe, where are the development sources that correct the problem, how
I can download the patch, etc...

I know that all these project are diferent communities, but if all
comunities make more easy to the end user some tasks, the users learns
more quickly how to contribute and work with the many projects and
tecknologies.

Don''t know how projects could make the thinks easy, but, in case of
netfilter, some tables explaining what matches are included in kernel (and
from what version) allow users to know what kernel version can use for
certain purposes.

For example, I know h323 where added into kernel at 2.6.17 kernel version,
but the community appears to be working into 2.6.16.x version to stabilize
it and and backported many bugfixes from 2.6.19.x to 2.6.16.x. I think
2.6.16.x kernel is the more stable at these days, but if I want h323
conntrack modules I can''t use pom-ńg because h323 is not in
pom-ng
today, and if I use old pom-ng, I know I will use a module with some bugs
corrected in 2.6.18.x kernel. Do I explained a problem that have many
users than me?

Another example, some body say in the lists that using "mark" will
allow
me to change routes by "mark value" or by "and-mark mask",
fine, that were
to substitute ROUTE tarjet. I think that is a correct solution but ...
What are the implications? 1) I can''t see any documentation in
netfilter
or iproute.
2) My distro don''t update the packages, I have to do it manually.
3) When I have an updated iproute package, kernel package, iptables
package and I finish and test the configuration, I have the knowledte to
write a little explanation/how-to to allow users to know how and don''t
waste their time looking for the steps ... some hours after I found a
wiki, I need to register, etc.... When I finished all the needed steps to
allow me to write a "more or less" public super-mini-howto I forgot
the
steps and I, finally, don''t write the info, because my systems are
working
for my purposes, I have my paper with my hand annotations and I need to
continue working in more tasks.

Sorry for the big e-mail, but it is only my user opinion about these type
o things.

Sorry for my poor english.

Regards

El Mie, 10 de Enero de 2007, 12:53, Jan Engelhardt
escribió:
> On Jan 10 2007 06:58, Patrick McHardy wrote:
>>Krzysztof Oledzki wrote:
>>>> Its still down, but the ROUTE patch is unmaintained anyway.
>>> How about attached (and inlined) patch. BTW - is it possible to add
a
Kconfig entry after a specific text, like with
Makefile.ladd?
>>> [POM-NG] ROUTE: 2.6.19 compatibility fix
>>> Make both IPv4 and IPv6 versions compatible with 2.6.19
>>Thanks Krzysztof, applied.
>>I would prefer to have someone maintain it externally though. Jan, are
you still interested in doing that? If you need help or webspace for an
external repository please let me know.
> I would give it a try. Though I would really prefer to have it in the
kernel and iptables rather than pomng or pomng-external. In
my
> opinion that simplifies maintainability. Changes in the netfilter API
seem to be the most common reason for patching (someone changed the
xt_match->match and xt_target->target signatures in 2.6.20 again!), and
keeping out-of-tree modules compiling with kernel-du-jour can be an
#ifdef pita. Then it''s really preferable to have 2.6.18 have a
xt_FOOBAR
with netfilter-2.6.18 signatures, and 2.6.20 with
> netfilter-2.6.20. Especially since many people run distributions with
RPM/DEBified iptables, so the POM `runme` will not be easy
to
> accomplish for the casual user. (I currently do have that issue - after
doing `svn up` on pomng, I have to manually move the changes to (my)
kernel rpm and (my) iptables rpm, because the days of `make install` are
GONE for me - at least I try.)
> I understand that POM does not require to compile with all
> kernels-of-the-last-three-months, but this also simplifies
> integration for end users. They do not need to backport/forward port
indated/outdated out-of-tree modules and, at best, do not even need to
recompile the kernel.
> Of course there are some modules that continue being out-of-tree because
they would not fit in (imagine a 500K geoip.c with a
> compiled-in big string array). Not sure what to do about them.
> Perhaps do it like chaostables [2.6.18-2.6.20], trying to keep it
working for a limited set of kernels.
> Oh well, that said, my ideal plan would be to get ROUTE TARPIT
> connlimit and u32 into mainline in one go, and perhaps, after review and
discussion, chaostables and some of the others that live
in
> Krzystof''s patchlet collection.
> Opinions, please?
> 	-`J''
> --

On Jan 10 2007 14:15, ArcosCom Linux User wrote:
>
>Another example, some body say in the lists that using "mark" will
allow
>me to change routes by "mark value" or by "and-mark
mask", fine, that were
>to substitute ROUTE tarjet. I think that is a correct solution but ...
>What are the implications? 1) I can''t see any documentation in
netfilter
>or iproute.
<rant>
So sad that it''s true. "ip r h" is possibly a complete BNF,
but
heck, even if I give a command that actually satisfies this BNF, I
still get an error like "RTNETLINK: Invalid argument" and
that''s it.
I really find this a showstopper. Nice shiny new technology which
can''t be used. At this time, -j ROUTE is *so* much simpler.

To make matters worse, I think netfilter, routing and QoS are too
separated (not that I can do much, too big a task for one man) -
unless you use classification with CLASSIFY in iptables, you have to
rematch all packets (in the style of "-p tcp") in tc egress.
</rant>



	-`J''
--

Sorry, a bit late replying to this but just wanted to add my thoughts.
> I think so, there are many old matches that are stables and I have to
> apply many times when I update the kernel. If they where into kernel and
> iptables (because they are now not as experimental than many months/years
> ago) these problem when new kernel releases and/or iptables releases
> disapears very quickly.
> 
> I have a great headache now, I had to patch my kernel, patch iptables,
> update iproute to allow "mark-and" operations for routing. Yes, I
can
> adapt many thinks and forgot many routing/filtering functionality, but
> then, my linux box will be useless for the purposes I deploy it.
> 
> I have no problem in patch and upgrade thinks, my problem is that I have
> no time to do all these steps every any important bug, improvement is
> released.
I would also like to see as many of the POM included in the stable
kernel. It''s a bit of a headache to patch in what I want each time I
update the kernel, and on a fresh system I have to install CURL just to
update POM just to add connlimit to the kernel...

It''s also a bit of a problem because I am looking to hand my server
over
to professional support. I''ve got to explain to them that if they ever
update the kernel that it will need patching. If they don''t, or forget,
then it will significantly affect the system''s performance in the
particular situation.

Finally, I can''t help but think it puts off newcomers. Although fairly
simple when you know what you''re doing, when someone comes from a
point-and-click windows background it is incredibly complicated!

The two I would like to see in the main kernel are connlimit (stable for
a few years?) and ipset.

Regards,

Andy Beverley

Andrew Beverley wrote:
> I would also like to see as many of the POM included in the stable
> kernel. It''s a bit of a headache to patch in what I want each time
I
> update the kernel, and on a fresh system I have to install CURL just to
> update POM just to add connlimit to the kernel...
IMHO, patching kernels to add some certain shiny-feature(TM) is
generally a bad idea if you don''t know how the patch internally works
or
if you can''t directly get support from the author of such patch.
Moreover, if the applied patchset is large, this can become a problem
since, at some point, you could report unreproducible bugs probably
introduced by one of your patches, thus annoying developers that are
worried about fixing bugs in the stable kernel branch asap. So, I would
suggest people to be more conservative when applying patches.

Anyway, if you think that some certain patch is stable enough to push it
forward to mainline, encourage the author to push it forward. Probably
there is a reason why he decided not to do that.

-- 
The dawn of the fourth age of Linux firewalling is coming; a time of
great struggle and heroic deeds -- J.Kadlecsik got inspired by J.Morris

On Wed, 2007-01-31 at 03:58 +0100, Pablo Neira Ayuso
wrote:
> Andrew Beverley wrote:
> > I would also like to see as many of the POM included in the stable
> > kernel. It''s a bit of a headache to patch in what I want each
time I
> > update the kernel, and on a fresh system I have to install CURL just
to
> > update POM just to add connlimit to the kernel...
> 
> IMHO, patching kernels to add some certain shiny-feature(TM) is
> generally a bad idea if you don''t know how the patch internally
works or
> if you can''t directly get support from the author of such patch.
Yes, agreed. I was more thinking of those that (look like) they have
been stable for a few years.
> Anyway, if you think that some certain patch is stable enough to push it
> forward to mainline, encourage the author to push it forward. Probably
> there is a reason why he decided not to do that.
Okay, I''ve emailed the author (of connlimit) but not received a reply.
I
did ask him a while ago on the same subject but didn''t really get a
reason as to why it is not. Anybody have any ideas?

In this case can *I* push it forward to the stable kernel?

Regards,

Andy Beverley

On Fri, 9 Feb 2007, Andrew Beverley wrote:
> On Wed, 2007-01-31 at 03:58 +0100, Pablo Neira Ayuso wrote:
>> Andrew Beverley wrote:
>>> I would also like to see as many of the POM included in the stable
>>> kernel. It''s a bit of a headache to patch in what I want
each time I
>>> update the kernel, and on a fresh system I have to install CURL
just to
>>> update POM just to add connlimit to the kernel...
>>
>> IMHO, patching kernels to add some certain shiny-feature(TM) is
>> generally a bad idea if you don''t know how the patch
internally works or
>> if you can''t directly get support from the author of such
patch.
>
> Yes, agreed. I was more thinking of those that (look like) they have
> been stable for a few years.
>
>> Anyway, if you think that some certain patch is stable enough to push
it
>> forward to mainline, encourage the author to push it forward. Probably
>> there is a reason why he decided not to do that.
>
> Okay, I''ve emailed the author (of connlimit) but not received a
reply. I
> did ask him a while ago on the same subject but didn''t really get
a
> reason as to why it is not. Anybody have any ideas?
>
> In this case can *I* push it forward to the stable kernel?
Please excuse me - I have been _extremely _ busy for the last three weeks.

Getting back to the question: generally I have no objection for forwarding 
connlinit to the mainline but I believe we should first investigate a 
possibilty to add support for other protocols than TCP. AFAIK at least UDP 
support could be very usefull - p2p software generates not only a lot of 
tcp cnnections but also udp flows and main job for this extension is to 
prevent conntrack database overflows.

BTW: I''m not the author of this code - I only volunteered to maintain
it.

Best regards,

 				Krzysztof Olędzki

Krzysztof Oledzki wrote:
> Getting back to the question: generally I have no objection for
> forwarding connlinit to the mainline but I believe we should first
> investigate a possibilty to add support for other protocols than TCP.
> AFAIK at least UDP support could be very usefull - p2p software
> generates not only a lot of tcp cnnections but also udp flows and main
> job for this extension is to prevent conntrack database overflows.
Feel free to post a version you consider suitable for merging
(without all the version ifdefs, only nf_conntrack support,
etc). I had a quick look at the current version and it seems
to maintain some internal hash of connections, IIRC that has
not always been the case. In case that change is from you
please add a short description. And it should probably support
all protocols.

> >>> I would also like to see as many of the POM included in the
stable
> >>> kernel. It''s a bit of a headache to patch in what I
want each time I
> >>> update the kernel, and on a fresh system I have to install
CURL just to
> >>> update POM just to add connlimit to the kernel...
> >>
> >> IMHO, patching kernels to add some certain shiny-feature(TM) is
> >> generally a bad idea if you don''t know how the patch
internally works or
> >> if you can''t directly get support from the author of such
patch.
> >
> > Yes, agreed. I was more thinking of those that (look like) they have
> > been stable for a few years.
> >
> >> Anyway, if you think that some certain patch is stable enough to
push it
> >> forward to mainline, encourage the author to push it forward.
Probably
> >> there is a reason why he decided not to do that.
> >
> > Okay, I''ve emailed the author (of connlimit) but not received
a reply. I
> > did ask him a while ago on the same subject but didn''t really
get a
> > reason as to why it is not. Anybody have any ideas?
> >
> > In this case can *I* push it forward to the stable kernel?
> 
> Please excuse me - I have been _extremely _ busy for the last three weeks.
No please accept my apologies - I was a bit impatient.
> Getting back to the question: generally I have no objection for forwarding 
> connlinit to the mainline but I believe we should first investigate a 
> possibilty to add support for other protocols than TCP. AFAIK at least UDP 
> support could be very usefull - p2p software generates not only a lot of 
> tcp cnnections but also udp flows and main job for this extension is to 
> prevent conntrack database overflows.
Very interesting. I had exactly the same thoughts myself, and have
actually already created a patch for hashlimit which matches on the
number of UDP ''connections''.

Of course, the problem with UDP is that there are no connections as such
to count, which is why I chose to patch hashlimit rather than connlimit.
Hashlimit (as I am sure you are aware) keeps a table of recent data
flows which die after a set time, making it easier to count UDP flows.
I''m not sure how easy this would be to achieve with connlimit.

I was planning on sending the patch to hashlimit''s author, if nothing
else just to get feedback on it, as it is the first kernel hacking I
have done. Maybe I should post it to the netfilter-devel list instead,
or am I using the wrong tool for the wrong job?

Regards,

Andy Beverley