Greetings from a newbie!
(Well, to this list anyway)

I'm having a problem and I hope someone here might be able to help...

I am strongly expecting an answer along the lines of "upgrade to
v2.4.x",=
 but=20
I would REALLY preffer to avoid that for now...

The setup:

"Home brewed" v2.2.24 (will patch to v2.2.25 later today) with the DS8
pa=
tch=20
applied. Currently downloading the DS9/rbtree/htb3 patches to be applied=20
later (obviously, unpatching the old DS8 first), and see if at least some=
 of=20
my problems go away.

Multiple cable/DSL lines with multiple default routes and equal cost=20
multipath.

The problems:

1) Ingress shaping/policing doesn't seem to work at all.

I haven't tried outgoing shaping/policing, but prioritizing traffic defin=
itely=20
works, and since activating it, my bandwidth usage has quite obviously ju=
mped=20
to closer to my available limits (from the bandwidth usage graphs) on=20
cable/DSL connections. ssh and ping latency has also gone through the flo=
or=20
after increasing the priority for those protocols (using a variant of the=
=20
wonderhaper). So that clearly seems to work.

When applying ingres shaping (policing filter) all executes fine without=20
reporting any errors, but

tc -s -d qdisc show dev eth1
and
tc -s -d filter show dev eth1

both say that no traffic has been caught by the rules (which just cannot =
be=20
right because I am using a u32 filter with src 0.0.0.0/0, and the same fi=
lter=20
works for outgoing traffic with dst 0.0.0.0/0.

Here is a script snippet:

tc qdisc add    dev $DEV ingress        \
                handle ffff:

tc filter add   dev $DEV                        \
                parent ffff:                    \
                protocol ip                     \
                prio 50                         \
                u32 match ip                    \
                src 0.0.0.0/0                   \
                police rate ${DOWNLINK}Kbit     \
                burst $[8*DOWNLINK]Kbit         \
                drop flowid :1


Can anyone hazard a guess as to why this is not doing what it should be? =
Is=20
this a know bug in DS8 and DS9 will fix it? I will try it anyway, just to=
=20
make sure, but some encouraging news would be nice. :-)

1.2) If the ingress traffic shaping is unfixable in v2.2.x, would it be=20
possible to instead set up a dummy network device, and set up an egress=20
shaper on the physical interface, and forward everything to the dummy=20
intgerface, and then use the dummy interface as the default gateway?=20
Effectively this would do the same thing as setting up two routers=20
bacl-to-back, and using only egress shaping on both routers to achieve=20
ingress shaping? Kind of like having a logical, rather than a physical=20
router?

2) ipmasqadm portfw unstable/unreliable

I have tried to use this approach to forward ports from the firewall to a=
n=20
internal server. It works OK initially, but within minutes, things start=20
going wrong. Some connections get through on one interface but not the ot=
her.=20
Later, connections from the same host will work on a different interface,=
 but=20
not the one it worked on initially.

This can be temporarily made to go away by doing
# ip route flush cache
a few times, but the problem will always return. After about a month of=20
uptime, this will get to the point where most connections fail on most=20
interfaces, and the only cure I have found is a reboot.

Connections that are actually physically proxied by using NetCat (where t=
he=20
source address doesn't need to be preserved) or redir (with --transproxy=20
where the source address does need to be preserved) work fine, and don't=20
experience the same problem at all.

Obviouly, I would prefer to use the "ipmasqadm portfw" solution
because i=
t is=20
massively more memory/CPU efficient than a generic user-space tcp proxy=20
deamon running from inetd.

3) teql vs. multiple default routes
Not really a problem, more of a matter of exploring options, and I haven'=
t=20
found any particularly detailed information on the subject.

Can a virtual load ballancing device be used to aggregate multiple ethern=
et=20
interfaces? Presumably load balancing of incoming things is still done in=
 the=20
traditional way (DNS+stuff), and a teql interface would only be used to l=
oad=20
balance connections initiated from the network itself?

Presumably, the advantage of using a teql interface over multiple default=
=20
routes is that load ballancing would be done per session, rather than per=
=20
route? Are there any routing issues from the external point of view with=20
using teql, or is it completely transparent to the outside world? What=20
concerns me is that the source IP address might be a private IP address o=
f=20
the teql interface. Are there any potential issues with this, or can stan=
dard=20
ipchains/masquerading rules be used to overcome any potential problems?

TIA.

Gordan

Hello,

On Thu, 20 Mar 2003, Gordan Bobic wrote:

> The setup:
>
> "Home brewed" v2.2.24 (will patch to v2.2.25 later today) with
the DS8 patch
> applied. Currently downloading the DS9/rbtree/htb3 patches to be applied
> later (obviously, unpatching the old DS8 first), and see if at least some
of
> my problems go away.

	Yes, DS-8 has many problems including security ones.
The only problem is that I still didn't upgraded the patches to
2.2.25, may be in the next days I'll find time to do so.

> Multiple cable/DSL lines with multiple default routes and equal cost
> multipath.

	I strongly recommend the route patches in such case:

http://www.ssi.bg/~ja/#routes-2.2
http://www.ssi.bg/~ja/routes-2.2.20-7.diff

> When applying ingres shaping (policing filter) all executes fine without
> reporting any errors, but
>
> tc -s -d qdisc show dev eth1
> and
> tc -s -d filter show dev eth1

	DS9 has fixes for the ingress stats

> Can anyone hazard a guess as to why this is not doing what it should be? Is
> this a know bug in DS8 and DS9 will fix it? I will try it anyway, just to
> make sure, but some encouraging news would be nice. :-)

	Yes, yes, I'll add it to the changelog

> 2) ipmasqadm portfw unstable/unreliable
>
> I have tried to use this approach to forward ports from the firewall to an
> internal server. It works OK initially, but within minutes, things start
> going wrong. Some connections get through on one interface but not the
other.
> Later, connections from the same host will work on a different interface,
but
> not the one it worked on initially.

	Such problems should be solved from the "routes" patches,
they will keep each traffic through its ISP.

	Read nano.txt from http://www.ssi.bg/~ja/#routes
It is for 2.4 but the concept and the routing rules are same.

> TIA.
>
> Gordan

Regards

--
Julian Anastasov <ja@ssi.bg>

On Thursday 20 Mar 2003 12:59, Julian Anastasov wrote:
> Hello,
>
> On Thu, 20 Mar 2003, Gordan Bobic wrote:
> > The setup:
> >
> > "Home brewed" v2.2.24 (will patch to v2.2.25 later today)
with the DS=
8
> > patch applied. Currently downloading the DS9/rbtree/htb3 patches to b=
e
> > applied later (obviously, unpatching the old DS8 first), and see if a=
t
> > least some of my problems go away.
>
> =09Yes, DS-8 has many problems including security ones.
> The only problem is that I still didn't upgraded the patches to
> 2.2.25, may be in the next days I'll find time to do so.

It all patched cleanly anyway, don't worry about it. I patched 2.2.24 and=
 then=20
patched that to 2.2.25. I haven't compiled it yet because I am not in fro=
nt=20
of the machine right now (not doing remote kernel upgrades - bad things=20
happen). :-)

> > Multiple cable/DSL lines with multiple default routes and equal cost
> > multipath.
>
> =09I strongly recommend the route patches in such case:
>
> http://www.ssi.bg/~ja/#routes-2.2
> http://www.ssi.bg/~ja/routes-2.2.20-7.diff

Aha! Thanks. I wonder if whis will cure my problems. BTW, that applies cl=
eanly=20
to my already heavily patched 2.2.25 tree.

> > When applying ingres shaping (policing filter) all executes fine with=
out
> > reporting any errors, but
> >
> > tc -s -d qdisc show dev eth1
> > and
> > tc -s -d filter show dev eth1
>
> =09DS9 has fixes for the ingress stats

Marvellous. :-)

> > Can anyone hazard a guess as to why this is not doing what it should =
be?
> > Is this a know bug in DS8 and DS9 will fix it? I will try it anyway, =
just
> > to make sure, but some encouraging news would be nice. :-)
>
> =09Yes, yes, I'll add it to the changelog

:-)

> > 2) ipmasqadm portfw unstable/unreliable
> >
> > I have tried to use this approach to forward ports from the firewall =
to
> > an internal server. It works OK initially, but within minutes, things
> > start going wrong. Some connections get through on one interface but =
not
> > the other. Later, connections from the same host will work on a diffe=
rent
> > interface, but not the one it worked on initially.
>
> =09Such problems should be solved from the "routes" patches,
> they will keep each traffic through its ISP.

Excellent. Note that I also use policy routing so that the sessions start=
ed to=20
ethX will always be reponded to on ethX. I think that is fairly standard=20
(things break horribly otherwise). However, what happens when a host rand=
omly=20
alternates between IP addresses it is starting sessions to/from? This sor=
t of=20
worked before, but it quickly started to break. I take it that with the=20
routes patch things will not fall apart like before?

> =09Read nano.txt from http://www.ssi.bg/~ja/#routes
> It is for 2.4 but the concept and the routing rules are same.

Thanks for that. I already have the setup working as far as multi-homedne=
ss=20
(that cannot possibly be a word...) is concerned, though.

Thanks for the help.

Gordan

Hello,

On Thu, 20 Mar 2003, Gordan Bobic wrote:

> > The only problem is that I still didn't upgraded the patches to
> > 2.2.25, may be in the next days I'll find time to do so.
>
> It all patched cleanly anyway, don't worry about it. I patched 2.2.24 and
then
> patched that to 2.2.25. I haven't compiled it yet because I am not in front

	OK, I see that skb_padto is used only from the net drivers,
so it is not fatal that skb_copy_grow will not copy skb->tc_index,
so I hope DS9 will work

> Excellent. Note that I also use policy routing so that the sessions started
to
> ethX will always be reponded to on ethX. I think that is fairly standard
> (things break horribly otherwise). However, what happens when a host
randomly
> alternates between IP addresses it is starting sessions to/from? This sort
of
> worked before, but it quickly started to break. I take it that with the
> routes patch things will not fall apart like before?

	Yes, the patch keeps each NAT connection bound to the
masquerade address and correctly uses the routing, usually for
multipath routes. Of course, in 2.2 a route cache flush leads to
much more problems (CPU cycles for rerouting) compared to 2.4.

Regards

--
Julian Anastasov <ja@ssi.bg>

Can this be done?

Provided the DS9 patch works for me, is it possible to somehow prioritize=
 what=20
is dropped first and what is dropped last? Something like attaching a=20
prio/cbq to the ingress qdisc? For example, I want SSH/POP3/IMAP4 traffic=
 to=20
be dropped last, and FTP/SMTP traffic to be dropped first. Can ingress=20
support in DS9 do this, or is it limited to indiscriminately dropping=20
traffic?

Regards.

Gordan

Hello,

On Thu, 20 Mar 2003, Gordan Bobic wrote:

> Provided the DS9 patch works for me, is it possible to somehow prioritize
what
> is dropped first and what is dropped last? Something like attaching a
> prio/cbq to the ingress qdisc? For example, I want SSH/POP3/IMAP4 traffic
to
> be dropped last, and FTP/SMTP traffic to be dropped first. Can ingress
> support in DS9 do this, or is it limited to indiscriminately dropping
> traffic?

	Except if someone has another idea, may be only dropping.
I'm not sure there is IMQ for 2.2 :) May be you can look at
Edge31-ca-u32 from the iproute2 examples, there are some ideas
about using policers. As for DS9, it tries to provide the same
interface to all qos objects as in 2.4, so it is possible to do
the same things as in 2.4, of course, there is one problem, nobody
wants to support new features in 2.2 :)

Regards

--
Julian Anastasov <ja@ssi.bg>

On Thursday 20 March 2003 23:36, Julian Anastasov wrote:
> Hello,
>
> On Thu, 20 Mar 2003, Gordan Bobic wrote:
> > Provided the DS9 patch works for me, is it possible to somehow
prioritize
> > what is dropped first and what is dropped last? Something like
attaching
> > a prio/cbq to the ingress qdisc? For example, I want SSH/POP3/IMAP4
> > traffic to be dropped last, and FTP/SMTP traffic to be dropped first.
Can
> > ingress support in DS9 do this, or is it limited to indiscriminately
> > dropping traffic?
>
> 	Except if someone has another idea, may be only dropping.
> I'm not sure there is IMQ for 2.2 :)

It doesn't appear to be there with the patches I applied... :-(

> May be you can look at
> Edge31-ca-u32 from the iproute2 examples, there are some ideas
> about using policers.

I have tried the ingress policing when the new patches, but according to my
mrtg graphs, it still doesn't seem to work, even in it's simplest form. I
THINK I saw it work with FTP, though (connection throttled), so I am not
sure. It could be that it works initially, then stops later. I will
investigate further...

> As for DS9, it tries to provide the same
> interface to all qos objects as in 2.4, so it is possible to do
> the same things as in 2.4, of course, there is one problem, nobody
> wants to support new features in 2.2 :)

Yeah, I know. I will upgrade at some point. ;-)

I seem to be having a problem with the newly applied patches, though. After
6-8 hours, I get a kernel crash. Black screen od death, hex dumps, Aiee,
killing interrupt handler, trying to kill idle task, etc. This happened
twice, so it seems unlikely to be a coincidence, as the only patches applied
were the DS9, rtree, HTB3 and routes. It used to be stable before. I am
hoping that it is ingress related stuff that kills it, so I have turned that
off (everything to do with ingress). We'll see if I get another crash now. If
not, than it would appear that it is ingress policing that kills it after a
while.

I will report back on this when I know more.

Regards.

Gordan

Hello,

On Sat, 22 Mar 2003, Gordan Bobic wrote:

> I seem to be having a problem with the newly applied patches, though. After
> 6-8 hours, I get a kernel crash. Black screen od death, hex dumps, Aiee,
> killing interrupt handler, trying to kill idle task, etc. This happened
> twice, so it seems unlikely to be a coincidence, as the only patches
applied
> were the DS9, rtree, HTB3 and routes. It used to be stable before. I am
> hoping that it is ingress related stuff that kills it, so I have turned
that
> off (everything to do with ingress). We'll see if I get another crash now.
If
> not, than it would appear that it is ingress policing that kills it after a
> while.
>
> I will report back on this when I know more.

	ok, it is interesting where is the oops exactly (may be
by avoiding modules and compiling qos in kernel). In any case,
I created patch to upgrade to HTB 3.10

> Regards.
>
> Gordan

Regards

--
Julian Anastasov <ja@ssi.bg>

On Sunday 23 March 2003 13:49, Julian Anastasov wrote:
> Hello,
>
> On Sat, 22 Mar 2003, Gordan Bobic wrote:
> > I seem to be having a problem with the newly applied patches, though.
> > After 6-8 hours, I get a kernel crash. Black screen od death, hex
dumps,
> > Aiee, killing interrupt handler, trying to kill idle task, etc. This
> > happened twice, so it seems unlikely to be a coincidence, as the only
> > patches applied were the DS9, rtree, HTB3 and routes. It used to be
> > stable before. I am hoping that it is ingress related stuff that kills
> > it, so I have turned that off (everything to do with ingress). We'll
see
> > if I get another crash now. If not, than it would appear that it is
> > ingress policing that kills it after a while.
> >
> > I will report back on this when I know more.
>
> 	ok, it is interesting where is the oops exactly (may be
> by avoiding modules and compiling qos in kernel). In any case,
> I created patch to upgrade to HTB 3.10

It would seem that the crash was DEFINITELY caused by ingress filtering. I
have switched it off (while keeping all egress qdiscs and filters), and now
the router has been up for 31 hours without a glitch. Oh, well. I will look
into patching with the latest HTB, but since I don't use it, there seems to
be little point...

Unfortunately, I cannot tell in which module the oops happens, because once
the machine crashes, there are no logs and the start of the error scrolls off
the screen. :-(

All I can tell for sure is that with ingress used it crashes, without it is
fine.

Regards.

Gordan

Hi.

I have just had another very similar crash. It would appear that it is not
related specifically to ingress policing. It appears to also happen using
just egress cbq/sfq. However, the uptime between crashes has increased from
around 6 hours to over 24 hours with ingress filtering switched off, with
similar (and probably heavier) usage.

Unfortunately, I can still find nothing useful in the logs, and there was
nothing useful on the screen either (just a hex dump).

I have had this problem (or a very similar one with the same symptoms) before
with QoS on earlier, unpatched v2.2.x kernels (circa 2.2.14 era). Has anybody
else experienced/reported anything similar?

I guess I'll just have to upgrade to v2.4...

Regards.

Gordan