thr3ads.net - LARTC - Ssh flow does not go into correct class. Help! [Jan 2005]

If this information is useful, please help other people find it:
Share via:

CcM

2005-Jan-26 10:56 UTC

Ssh flow does not go into correct class. Help!

I''m a new comer. I have problems using tc+htb. I run the following
commands, and expect outgoing ssh flow goes into 1:11. But actually it goes into
default 12. What''s wrong?
 
tc qdisc add dev eth0 root handle 1: htb default 12
 tc class add dev eth0 parent 1: classid 1:1 htb rate 1000kbit ceil 2000kbit
prio 1
tc class add dev eth0 parent 1:1 classid 1:11 htb rate 100kbit ceil 100kbit prio
1
tc class add dev eth0 parent 1:1 classid 1:12 htb rate 800kbit ceil 1600kbit
prio 1
tc class add dev eth0 parent 1:1 classid 1:13 htb rate 100kbit ceil 100kbit prio
1
tc qdisc add dev eth0 parent 1:11 handle 111: sfq perturb 5
tc qdisc add dev eth0 parent 1:12 handle 112: sfq perturb 5
tc qdisc add dev eth0 parent 1:13 handle 113: sfq perturb 5
 
iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22 -j MARK --set-mark 20
iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22 -j RETURN
 
tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw flowid 1:11

Thanks a lot!
CcM




---------------------------------
Do You Yahoo!?
注册世界一流品质的雅虎免费电邮
--0-918797084-1106736984=:17715
Content-Type: text/html; charset=gb2312
Content-Transfer-Encoding: 8bit

<DIV>I''m a new comer. I have problems using tc+htb. I run the
following commands, and expect outgoing ssh flow goes into 1:11. But actually it
goes into default 12. What''s wrong?</DIV>
<DIV>&nbsp;</DIV>
<DIV>tc qdisc add dev eth0 root handle 1: htb default
12<BR>&nbsp;tc class add dev eth0 parent 1: classid 1:1 htb rate
1000kbit ceil 2000kbit prio 1<BR>tc class add dev eth0 parent 1:1 classid
1:11 htb rate 100kbit ceil 100kbit prio 1<BR>tc class add dev eth0 parent
1:1 classid 1:12 htb rate 800kbit ceil 1600kbit prio 1<BR>tc class add dev
eth0 parent 1:1 classid 1:13 htb rate 100kbit ceil 100kbit prio 1<BR>tc
qdisc add dev eth0 parent 1:11 handle 111: sfq perturb 5<BR>tc qdisc add
dev eth0 parent 1:12 handle 112: sfq perturb 5<BR>tc qdisc add dev eth0
parent 1:13 handle 113: sfq perturb 5</DIV>
<DIV>&nbsp;</DIV>
<DIV>iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22 -j MARK
--set-mark 20<BR>iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22
-j RETURN</DIV>
<DIV>&nbsp;</DIV>
<DIV>tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw
flowid 1:11<BR></DIV>
<DIV>
<DIV>Thanks a lot!</DIV>
<DIV>CcM</DIV></DIV><p><br><hr size=1>
<b>Do You Yahoo!?</b><br>
<a
href="http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/"
target=blank>注册世界一流品质的雅虎免费电邮</a>
--0-918797084-1106736984=:17715--
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

CcM

2005-Jan-26 11:06 UTC

head link

Ssh flow does not go into correct class. Help!

I''m a new comer. I have problems using tc+htb. I run the following
commands, and expect outgoing ssh flow goes into 1:11. But actually it goes into
default 12. What''s wrong?
 
tc qdisc add dev eth0 root handle 1: htb default 12
 tc class add dev eth0 parent 1: classid 1:1 htb rate 1000kbit ceil 2000kbit
prio 1
tc class add dev eth0 parent 1:1 classid 1:11 htb rate 100kbit ceil 100kbit prio
1
tc class add dev eth0 parent 1:1 classid 1:12 htb rate 800kbit ceil 1600kbit
prio 1
tc class add dev eth0 parent 1:1 classid 1:13 htb rate 100kbit ceil 100kbit prio
1
tc qdisc add dev eth0 parent 1:11 handle 111: sfq perturb 5
tc qdisc add dev eth0 parent 1:12 handle 112: sfq perturb 5
tc qdisc add dev eth0 parent 1:13 handle 113: sfq perturb 5
 
iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22 -j MARK --set-mark 20
iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22 -j RETURN
 
tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw flowid 1:11

Thanks a lot!
CcM






---------------------------------
Do You Yahoo!?
注册世界一流品质的雅虎免费电邮
--0-209373516-1106737605=:67202
Content-Type: text/html; charset=gb2312
Content-Transfer-Encoding: 8bit

<DIV>
<DIV>
<DIV>I''m a new comer. I have problems using tc+htb. I run the
following commands, and expect outgoing ssh flow goes into 1:11. But actually it
goes into default 12. What''s wrong?</DIV>
<DIV>&nbsp;</DIV>
<DIV>tc qdisc add dev eth0 root handle 1: htb default
12<BR>&nbsp;tc class add dev eth0 parent 1: classid 1:1 htb rate
1000kbit ceil 2000kbit prio 1<BR>tc class add dev eth0 parent 1:1 classid
1:11 htb rate 100kbit ceil 100kbit prio 1<BR>tc class add dev eth0 parent
1:1 classid 1:12 htb rate 800kbit ceil 1600kbit prio 1<BR>tc class add dev
eth0 parent 1:1 classid 1:13 htb rate 100kbit ceil 100kbit prio 1<BR>tc
qdisc add dev eth0 parent 1:11 handle 111: sfq perturb 5<BR>tc qdisc add
dev eth0 parent 1:12 handle 112: sfq perturb 5<BR>tc qdisc add dev eth0
parent 1:13 handle 113: sfq perturb 5</DIV>
<DIV>&nbsp;</DIV>
<DIV>iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22 -j MARK
--set-mark 20<BR>iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22
-j RETURN</DIV>
<DIV>&nbsp;</DIV>
<DIV>tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw
flowid 1:11<BR></DIV>
<DIV>
<DIV>Thanks a lot!</DIV>
<DIV>CcM</DIV></DIV></DIV></DIV><p><br><hr
size=1>
<b>Do You Yahoo!?</b><br>
<a
href="http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/"
target=blank>注册世界一流品质的雅虎免费电邮</a>
--0-209373516-1106737605=:67202--
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

CcM

2005-Jan-26 11:45 UTC

head link

Re: Ssh flow does not go into correct class. Help!

Sorry, I did not make it clear. The machine I want to do traffic control is a
server which run sshd. I think it should be --sport.
I also tried to use :
        iptables -t mangle -A PREROUTING -m tos --tos Minimize-Delay -j MARK
--set-mark 20
the ssh traffice still goes into default 12.
 
following is the evidence:
bash#tc -s -d class show dev eth0

class htb 1:11 parent 1:1 leaf 111: prio 1 quantum 1000 rate 10Kbit ceil 10Kbit
burst 1611b/8 mpu 0b cburst 1611b/8 mpu 0b level 0
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0) 
 lended: 0 borrowed: 0 giants: 0
 tokens: 1031680 ctokens: 1031680
 
class htb 1:1 root rate 1000Kbit ceil 2000Kbit burst 2879b/8 mpu 0b cburst
4159b/8 mpu 0b level 7
 Sent 21163 bytes 149 pkts (dropped 0, overlimits 0) 
 rate 5bps 
 lended: 0 borrowed: 0 giants: 0
 tokens: 17818 ctokens: 13004
 
class htb 1:13 parent 1:1 leaf 113: prio 1 quantum 1280 rate 100Kbit ceil
100Kbit burst 1727b/8 mpu 0b cburst 1727b/8 mpu 0b level 0
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0) 
 lended: 0 borrowed: 0 giants: 0
 tokens: 110592 ctokens: 110592
 
class htb 1:12 parent 1:1 leaf 112: prio 1 quantum 10240 rate 800Kbit ceil
1600Kbit burst 2623b/8 mpu 0b cburst 3647b/8 mpu 0b level 0
 Sent 21163 bytes 149 pkts (dropped 0, overlimits 0) 
 rate 5bps 
 lended: 149 borrowed: 0 giants: 0
 tokens: 20225 ctokens: 14208



Mika Hirvonen <hirvox@gmail.com> wrote:
On Wed, 26 Jan 2005 18:56:24 +0800 (CST), CcM wrote:> I''m a new comer. I have problems using tc+htb. I run the following
commands,
> and expect outgoing ssh flow goes into 1:11. But actually it goes into
> default 12. What''s wrong? 
> iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22 -j MARK
--set-mark
> 20
> iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 22 -j RETURN 
These two lines should use --dport instead of --sport. The source port
is random.

AFAIR OpenSSH (and probably others) automatically mark their packets
with proper ToS flags, so you should considering using those instead
of port numbers. If you classify ssh traffic by port number only, SSH
file transfers (scp and sftp) end up in the same class as interactive
traffic.


-- 
Mika Hirvonen 




---------------------------------
Do You Yahoo!?
注册世界一流品质的雅虎免费电邮
--0-1327050722-1106739928=:74122
Content-Type: text/html; charset=gb2312
Content-Transfer-Encoding: 8bit

<DIV>
<DIV>Sorry, I did not make it clear. The machine I want to do traffic
control is a server which run sshd. I think it should be --sport. </DIV>
<DIV>I also tried to use :</DIV>
<DIV>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
iptables -t mangle -A PREROUTING -m tos --tos Minimize-Delay -j MARK --set-mark
20</DIV>
<DIV>the ssh traffice still goes into default 12.</DIV>
<DIV>&nbsp;</DIV>
<DIV>following is the evidence:</DIV>
<DIV>bash#tc -s -d class show dev eth0</DIV>
<DIV><BR>class htb 1:11 parent 1:1 leaf 111: prio 1 quantum 1000
rate 10Kbit ceil 10Kbit burst 1611b/8 mpu 0b cburst 1611b/8 mpu 0b level 0
<BR>&nbsp;Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
<BR>&nbsp;lended: 0 borrowed: 0 giants: 0<BR>&nbsp;tokens:
1031680 ctokens: 1031680</DIV>
<DIV>&nbsp;</DIV>
<DIV>class htb 1:1 root rate 1000Kbit ceil 2000Kbit burst 2879b/8 mpu 0b
cburst 4159b/8 mpu 0b level 7 <BR>&nbsp;Sent 21163 bytes 149 pkts
(dropped 0, overlimits 0) <BR>&nbsp;rate 5bps
<BR>&nbsp;lended: 0 borrowed: 0 giants: 0<BR>&nbsp;tokens:
17818 ctokens: 13004</DIV>
<DIV>&nbsp;</DIV>
<DIV>class htb 1:13 parent 1:1 leaf 113: prio 1 quantum 1280 rate 100Kbit
ceil 100Kbit burst 1727b/8 mpu 0b cburst 1727b/8 mpu 0b level 0
<BR>&nbsp;Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
<BR>&nbsp;lended: 0 borrowed: 0 giants: 0<BR>&nbsp;tokens:
110592 ctokens: 110592</DIV>
<DIV>&nbsp;</DIV>
<DIV>class htb 1:12 parent 1:1 leaf 112: prio 1 quantum 10240 rate 800Kbit
ceil 1600Kbit burst 2623b/8 mpu 0b cburst 3647b/8 mpu 0b level 0
<BR>&nbsp;Sent 21163 bytes 149 pkts (dropped 0, overlimits 0)
<BR>&nbsp;rate 5bps <BR>&nbsp;lended: 149 borrowed: 0
giants: 0<BR>&nbsp;tokens: 20225 ctokens: 14208<BR></DIV>
<DIV><BR><BR><B><I>Mika Hirvonen
&lt;hirvox@gmail.com&gt;</I></B> wrote:</DIV>
<BLOCKQUOTE class=replbq style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px;
BORDER-LEFT: #1010ff 2px solid">On Wed, 26 Jan 2005 18:56:24 +0800
(CST), CcM <BEATLELITTLE@YAHOO.COM.CN>wrote:<BR>&gt;
I''m a new comer. I have problems using tc+htb. I run the following
commands,<BR>&gt; and expect outgoing ssh flow goes into 1:11. But
actually it goes into<BR>&gt; default 12. What''s wrong?
<BR><BR>&gt; iptables -t mangle -A PREROUTING -p tcp -m tcp
--sport 22 -j MARK --set-mark<BR>&gt; 20<BR>&gt; iptables -t
mangle -A PREROUTING -p tcp -m tcp --sport 22 -j RETURN
<BR><BR>These two lines should use --dport instead of --sport. The
source port<BR>is random.<BR><BR>AFAIR OpenSSH (and probably
others) automatically mark their packets<BR>with proper ToS flags, so you
should considering using those instead<BR>of port numbers. If you classify
ssh traffic by port number only, SSH<BR>file transfers (scp and sft
 p) end up in the same class as
interactive<BR>traffic.<BR><BR><BR>-- <BR>Mika
Hirvonen
 <HIRVOX@GMAIL.COM><BR></BLOCKQUOTE></DIV><p><br><hr
size=1>
<b>Do You Yahoo!?</b><br>
<a
href="http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/"
target=blank>注册世界一流品质的雅虎免费电邮</a>
--0-1327050722-1106739928=:74122--
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Catalin(ux aka Dino) BOIE

2005-Jan-26 11:58 UTC

head link

Re: Ssh flow does not go into correct class. Help!

Use:
iptables -t mangle -A POSTROUTING -p tcp -m tcp --sport 22 -j MARK \
 	--set-mark 20

---
Catalin(ux aka Dino) BOIE
catab at deuroconsult.ro
http://kernel.umbrella.ro/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

CcM

2005-Jan-26 12:20 UTC

head link

Re: Ssh flow does not go into correct class. Help!

It does work! Thanks!
But many articles write like what I did, what''s the difference between
POSTROUTING and PREROUTING. I mean I know these are two hook point in ip stack,
but why should be POSTROUTING here when do mangle?

"Catalin(ux aka Dino) BOIE" <util@deuroconsult.ro> wrote:
Use:
iptables -t mangle -A POSTROUTING -p tcp -m tcp --sport 22 -j MARK \
--set-mark 20

---
Catalin(ux aka Dino) BOIE
catab at deuroconsult.ro
http://kernel.umbrella.ro/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/



---------------------------------
Do You Yahoo!?
注册世界一流品质的雅虎免费电邮
--0-932001211-1106742050=:92945
Content-Type: text/html; charset=gb2312
Content-Transfer-Encoding: 8bit

<DIV>It does work! Thanks!</DIV>
<DIV>But many articles write like what I did, what''s the
difference between POSTROUTING and PREROUTING. I mean I know these are two hook
point in ip stack, but why should be POSTROUTING&nbsp;here when do
mangle?&nbsp;<BR><BR><B><I>"Catalin(ux aka
Dino) BOIE" &lt;util@deuroconsult.ro&gt;</I></B>
wrote:</DIV>
<BLOCKQUOTE class=replbq style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px;
BORDER-LEFT: #1010ff 2px solid">Use:<BR>iptables -t mangle -A
POSTROUTING -p tcp -m tcp --sport 22 -j MARK \<BR>--set-mark
20<BR><BR>---<BR>Catalin(ux aka Dino) BOIE<BR>catab at
deuroconsult.ro<BR>http://kernel.umbrella.ro/<BR>_______________________________________________<BR>LARTC
mailing list /
LARTC@mailman.ds9a.nl<BR>http://mailman.ds9a.nl/mailman/listinfo/lartc
HOWTO: http://lartc.org/<BR></BLOCKQUOTE><p><br><hr
size=1>
<b>Do You Yahoo!?</b><br>
<a
href="http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/"
target=blank>注册世界一流品质的雅虎免费电邮</a>
--0-932001211-1106742050=:92945--
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Catalin(ux aka Dino) BOIE

2005-Jan-26 12:42 UTC

head link

Re: Ssh flow does not go into correct class. Help!

CcM

2005-Jan-27 01:57 UTC

head link

Re: Ssh flow does not go into correct class. Help!

Now I want to control ports with ips specified, like following, it seems not
work, only --sport effect. Ftp-data goes to ips beside 10.190.6.16 also goes
into 1:12 while default is 1:13. Why?
 
tc qdisc add dev eth0 root handle 1: htb default 13
tc class add dev eth0 parent 1: classid 1:1 htb rate 80000kbit ceil 80000kbit
prio 1
tc class add dev eth0 parent 1:1 classid 1:11 htb rate 40kbit ceil 40kbit prio 1
tc class add dev eth0 parent 1:1 classid 1:12 htb rate 800kbit ceil 1600kbit
prio 1
tc class add dev eth0 parent 1:1 classid 1:13 htb rate 70000kbit ceil 80000kbit
prio 1
tc qdisc add dev eth0 parent 1:11 handle 111: sfq perturb 5
tc qdisc add dev eth0 parent 1:12 handle 112: sfq perturb 5
tc qdisc add dev eth0 parent 1:13 handle 113: sfq perturb 5
 
iptables -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j MARK --set-mark 20
iptables -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j RETURN

iptables -t mangle -A POSTROUTING -d 10.190.6.16  -p tcp --sport ftp-data -j
MARK --set-mark 21
iptables -t mangle -A POSTROUTING -d 10.190.6.16 -p tcp --sport ftp-data -j
RETURN
 
tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw flowid 1:11
tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 21 fw flowid 1:12


"Catalin(ux aka Dino) BOIE" <util@deuroconsult.ro> wrote:
On Wed, 26 Jan 2005, CcM wrote:
> It does work! Thanks!
> But many articles write like what I did, what''s the difference
between POSTROUTING and PREROUTING. I mean I know these are two hook point in ip
stack, but why should be POSTROUTING here when do mangle?
>
> "Catalin(ux aka Dino) BOIE" wrote:
> Use:
> iptables -t mangle -A POSTROUTING -p tcp -m tcp --sport 22 -j MARK \
> --set-mark 20
Because local generated packets doesn''t go through PREROUTING.
But, best is to use OUTPUT.
>
> ---
> Catalin(ux aka Dino) BOIE
> catab at deuroconsult.ro
> http://kernel.umbrella.ro/
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
>
>
> ---------------------------------
> Do You Yahoo!?
> 注册世界一流品质的雅虎免费电邮
---
Catalin(ux aka Dino) BOIE
catab at deuroconsult.ro
http://kernel.umbrella.ro/


---------------------------------
Do You Yahoo!?
注册世界一流品质的雅虎免费电邮
--0-1558822556-1106791077=:86193
Content-Type: text/html; charset=gb2312
Content-Transfer-Encoding: 8bit

<DIV>Now I want to control ports with ips specified, like following, it
seems&nbsp;not work, only&nbsp;--sport effect. Ftp-data goes to
ips&nbsp;beside 10.190.6.16 also goes into 1:12 while default is 1:13.
Why?</DIV>
<DIV>&nbsp;</DIV>
<DIV>tc qdisc add dev eth0 root handle 1: htb default 13<BR>tc class
add dev eth0 parent 1: classid 1:1 htb rate 80000kbit ceil 80000kbit prio
1<BR>tc class add dev eth0 parent 1:1 classid 1:11 htb rate 40kbit ceil
40kbit prio 1<BR>tc class add dev eth0 parent 1:1 classid 1:12 htb rate
800kbit ceil 1600kbit prio 1<BR>tc class add dev eth0 parent 1:1 classid
1:13 htb rate 70000kbit ceil 80000kbit prio 1<BR>tc qdisc add dev eth0
parent 1:11 handle 111: sfq perturb 5<BR>tc qdisc add dev eth0 parent 1:12
handle 112: sfq perturb 5<BR>tc qdisc add dev eth0 parent 1:13 handle 113:
sfq perturb 5</DIV>
<DIV>&nbsp;</DIV>
<DIV>iptables -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j MARK
--set-mark 20<BR>iptables -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j
RETURN</DIV>
<DIV><BR>iptables -t mangle -A POSTROUTING -d 10.190.6.16&nbsp;
-p tcp --sport ftp-data -j MARK --set-mark 21<BR>iptables -t mangle -A
POSTROUTING -d 10.190.6.16 -p tcp --sport ftp-data -j
RETURN<BR>&nbsp;</DIV>
<DIV>tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw
flowid 1:11<BR>tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle
21 fw flowid
1:12<BR><BR><BR><B><I>"Catalin(ux aka Dino)
BOIE" &lt;util@deuroconsult.ro&gt;</I></B>
wrote:</DIV>
<BLOCKQUOTE class=replbq style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px;
BORDER-LEFT: #1010ff 2px solid">On Wed, 26 Jan 2005, CcM
wrote:<BR><BR>&gt; It does work! Thanks!<BR>&gt; But
many articles write like what I did, what''s the difference between
POSTROUTING and PREROUTING. I mean I know these are two hook point in ip stack,
but why should be POSTROUTING here when do
mangle?<BR>&gt;<BR>&gt; "Catalin(ux aka Dino)
BOIE" <UTIL@DEUROCONSULT.RO>wrote:<BR>&gt;
Use:<BR>&gt; iptables -t mangle -A POSTROUTING -p tcp -m tcp --sport
22 -j MARK \<BR>&gt; --set-mark 20<BR><BR>Because local
generated packets doesn''t go through PREROUTING.<BR>But, best is
to use OUTPUT.<BR><BR>&gt;<BR>&gt;
---<BR>&gt; Catalin(ux aka Dino) BOIE<BR>&gt; catab at
deuroconsult.ro<BR>&gt; http://kernel.umbrella.ro/<BR>&gt;
_______________________________________________<BR>&gt; LARTC mailing
list / LARTC@mailman.ds9a.nl<BR>&
 gt; http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
 http://lartc.org/<BR>&gt;<BR>&gt;<BR>&gt;<BR>&gt;
---------------------------------<BR>&gt; Do You
Yahoo!?<BR>&gt;
注册世界一流品质的雅虎免费电邮<BR><BR>---<BR>Catalin(ux aka Dino)
BOIE<BR>catab at
deuroconsult.ro<BR>http://kernel.umbrella.ro/</BLOCKQUOTE><p><br><hr
size=1>
<b>Do You Yahoo!?</b><br>
<a
href="http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/"
target=blank>注册世界一流品质的雅虎免费电邮</a>
--0-1558822556-1106791077=:86193--
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Catalin(ux aka Dino) BOIE

2005-Jan-27 07:47 UTC

head link

Re: Ssh flow does not go into correct class. Help!

CcM

2005-Jan-28 01:44 UTC

head link

Re: Ssh flow does not go into correct class. Help!

Look at this case, it even does no work!
 
(following command run on ssh server.)
tc qdisc add dev eth0 root handle 1: htb default 13
tc class add dev eth0 parent 1: classid 1:1 htb rate 500kbit ceil 500kbit
tc class add dev eth0 parent 1:1 classid 1:11 htb rate 100kbit ceil 500kbit prio
1
tc class add dev eth0 parent 1:1 classid 1:12 htb rate 100kbit ceil 500kbit prio
1
tc class add dev eth0 parent 1:1 classid 1:13 htb rate 100kbit ceil 500kbit prio
1
tc qdisc add dev eth0 parent 1:11 handle 111: sfq perturb 5
tc qdisc add dev eth0 parent 1:12 handle 112: sfq perturb 5
tc qdisc add dev eth0 parent 1:13 handle 113: sfq perturb 5
 
iptables -t mangle -A OUTPUT -d 10.190.6.16 -j MARK --set-mark 20
iptables -t mangle -A OUTPUT -d 10.190.6.16 -j RETURN
 
tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw flowid 1:11

I expect data from ssh server to my client 10.190.6.16 should goes into 1:11,
and other data goes into default 1:13. But when I use ssh client on other
machine(not 10.190.6.16), the data seems also goes into 1:11(I got this from tc
-s -d class show dev eth0). Why?

"Catalin(ux aka Dino) BOIE" <util@deuroconsult.ro> wrote:
On Thu, 27 Jan 2005, CcM wrote:
> Now I want to control ports with ips specified, like following, it seems
not work, only --sport effect. Ftp-data goes to ips beside 10.190.6.16 also goes
into 1:12 while default is 1:13. Why?
>
> tc qdisc add dev eth0 root handle 1: htb default 13
> tc class add dev eth0 parent 1: classid 1:1 htb rate 80000kbit ceil
80000kbit prio 1
> tc class add dev eth0 parent 1:1 classid 1:11 htb rate 40kbit ceil 40kbit
prio 1
> tc class add dev eth0 parent 1:1 classid 1:12 htb rate 800kbit ceil
1600kbit prio 1
> tc class add dev eth0 parent 1:1 classid 1:13 htb rate 70000kbit ceil
80000kbit prio 1
> tc qdisc add dev eth0 parent 1:11 handle 111: sfq perturb 5
> tc qdisc add dev eth0 parent 1:12 handle 112: sfq perturb 5
> tc qdisc add dev eth0 parent 1:13 handle 113: sfq perturb 5
>
> iptables -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j MARK --set-mark 20
> iptables -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j RETURN
>
> iptables -t mangle -A POSTROUTING -d 10.190.6.16 -p tcp --sport ftp-data -j
MARK --set-mark 21
> iptables -t mangle -A POSTROUTING -d 10.190.6.16 -p tcp --sport ftp-data -j
RETURN
I bet you use passive mode in ftp so you get a weird port, not ftp-data 
(20). ftp is tricky to match. You can use this:
iptables -t mangle -A POSTROUTING -d x.x.x.x -m helper --helper ftp \
-j MARK --set-mark 21
This will mark with 21 all ftp control+data connections.
>
> tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw flowid
1:11
> tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 21 fw flowid
1:12
>
>
> "Catalin(ux aka Dino) BOIE" wrote:
> On Wed, 26 Jan 2005, CcM wrote:
>
>> It does work! Thanks!
>> But many articles write like what I did, what''s the difference
between POSTROUTING and PREROUTING. I mean I know these are two hook point in ip
stack, but why should be POSTROUTING here when do mangle?
>>
>> "Catalin(ux aka Dino) BOIE" wrote:
>> Use:
>> iptables -t mangle -A POSTROUTING -p tcp -m tcp --sport 22 -j MARK \
>> --set-mark 20
>
> Because local generated packets doesn''t go through PREROUTING.
> But, best is to use OUTPUT.
>
>>
>> ---
>> Catalin(ux aka Dino) BOIE
>> catab at deuroconsult.ro
>> http://kernel.umbrella.ro/
>> _______________________________________________
>> LARTC mailing list / LARTC@mailman.ds9a.nl
>> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>>
>>
>>
>> ---------------------------------
>> Do You Yahoo!?
>> 注册世界一流品质的雅虎免费电邮
>
> ---
> Catalin(ux aka Dino) BOIE
> catab at deuroconsult.ro
> http://kernel.umbrella.ro/
>
>
> ---------------------------------
> Do You Yahoo!?
> 注册世界一流品质的雅虎免费电邮
---
Catalin(ux aka Dino) BOIE
catab at deuroconsult.ro
http://kernel.umbrella.ro/


---------------------------------
Do You Yahoo!?
注册世界一流品质的雅虎免费电邮
--0-1665266126-1106876660=:42398
Content-Type: text/html; charset=gb2312
Content-Transfer-Encoding: 8bit

<DIV>Look at this case, it even does no work!</DIV>
<DIV>&nbsp;</DIV>
<DIV>(following command run on ssh server.)</DIV>
<DIV><EM>tc qdisc add dev eth0 root handle 1: htb default
13<BR>tc class add dev eth0 parent 1: classid 1:1 htb rate 500kbit ceil
500kbit<BR>tc class add dev eth0 parent 1:1 classid 1:11 htb rate 100kbit
ceil 500kbit prio 1<BR>tc class add dev eth0 parent 1:1 classid 1:12 htb
rate 100kbit ceil 500kbit prio 1<BR>tc class add dev eth0 parent 1:1
classid 1:13 htb rate 100kbit ceil 500kbit prio 1<BR>tc qdisc add dev eth0
parent 1:11 handle 111: sfq perturb 5<BR>tc qdisc add dev eth0 parent 1:12
handle 112: sfq perturb 5<BR>tc qdisc add dev eth0 parent 1:13 handle 113:
sfq perturb 5</EM></DIV>
<DIV><EM></EM>&nbsp;</DIV>
<DIV><EM>iptables -t mangle -A OUTPUT -d 10.190.6.16 -j MARK
--set-mark 20<BR>iptables -t mangle -A OUTPUT -d 10.190.6.16 -j
RETURN</EM></DIV>
<DIV><EM></EM>&nbsp;</DIV>
<DIV><EM>tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle
20 fw flowid 1:11<BR></EM><BR>I expect data from ssh server to
my client 10.190.6.16 should goes into 1:11,&nbsp;and&nbsp;other data
goes into default 1:13. But when I use ssh client on other machine(not
10.190.6.16), the data seems also goes into 1:11(I got this from tc -s -d class
show dev eth0). Why?</DIV>
<DIV><BR><B><I>"Catalin(ux aka Dino) BOIE"
&lt;util@deuroconsult.ro&gt;</I></B> wrote:</DIV>
<BLOCKQUOTE class=replbq style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px;
BORDER-LEFT: #1010ff 2px solid">On Thu, 27 Jan 2005, CcM
wrote:<BR><BR>&gt; Now I want to control ports with ips
specified, like following, it seems not work, only --sport effect. Ftp-data goes
to ips beside 10.190.6.16 also goes into 1:12 while default is 1:13.
Why?<BR>&gt;<BR>&gt; tc qdisc add dev eth0 root handle 1:
htb default 13<BR>&gt; tc class add dev eth0 parent 1: classid 1:1 htb
rate 80000kbit ceil 80000kbit prio 1<BR>&gt; tc class add dev eth0
parent 1:1 classid 1:11 htb rate 40kbit ceil 40kbit prio 1<BR>&gt; tc
class add dev eth0 parent 1:1 classid 1:12 htb rate 800kbit ceil 1600kbit prio
1<BR>&gt; tc class add dev eth0 parent 1:1 classid 1:13 htb rate
70000kbit ceil 80000kbit prio 1<BR>&gt; tc qdisc add dev eth0 parent
1:11 handle 111: sfq perturb 5<BR>&gt; tc qdisc add dev eth0 parent
1:12 handle 112: sfq
 perturb 5<BR>&gt; tc qdisc add dev eth0 parent 1:13 handle 113: sfq
perturb 5<BR>&gt;<BR>&
 gt;
 iptables -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j MARK --set-mark
20<BR>&gt; iptables -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j
RETURN<BR>&gt;<BR>&gt; iptables -t mangle -A POSTROUTING -d
10.190.6.16 -p tcp --sport ftp-data -j MARK --set-mark 21<BR>&gt;
iptables -t mangle -A POSTROUTING -d 10.190.6.16 -p tcp --sport ftp-data -j
RETURN<BR><BR>I bet you use passive mode in ftp so you get a weird
port, not ftp-data <BR>(20). ftp is tricky to match. You can use
this:<BR>iptables -t mangle -A POSTROUTING -d x.x.x.x -m helper --helper
ftp \<BR>-j MARK --set-mark 21<BR>This will mark with 21 all ftp
control+data connections.<BR><BR>&gt;<BR>&gt; tc
filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw flowid
1:11<BR>&gt; tc filter add dev eth0 parent 1:0 prio 1 protocol ip
handle 21 fw flowid 1:12<BR>&gt;<BR>&gt;<BR>&gt;
"Catalin(ux aka Dino) BOIE" <UTIL@DEUROCONSULT.RO>wr
 ote:<BR>&gt; On Wed, 26 Jan 2005, CcM
wrote:<BR>&gt;<BR>&gt;&gt; It does work!
Thanks!<BR>
 &gt;&gt;
 But many articles write like what I did, what''s the difference between
POSTROUTING and PREROUTING. I mean I know these are two hook point in ip stack,
but why should be POSTROUTING here when do
mangle?<BR>&gt;&gt;<BR>&gt;&gt; "Catalin(ux aka
Dino) BOIE" wrote:<BR>&gt;&gt; Use:<BR>&gt;&gt;
iptables -t mangle -A POSTROUTING -p tcp -m tcp --sport 22 -j MARK
\<BR>&gt;&gt; --set-mark 20<BR>&gt;<BR>&gt;
Because local generated packets doesn''t go through
PREROUTING.<BR>&gt; But, best is to use
OUTPUT.<BR>&gt;<BR>&gt;&gt;<BR>&gt;&gt;
---<BR>&gt;&gt; Catalin(ux aka Dino)
BOIE<BR>&gt;&gt; catab at
deuroconsult.ro<BR>&gt;&gt;
http://kernel.umbrella.ro/<BR>&gt;&gt;
_______________________________________________<BR>&gt;&gt; LARTC
mailing list / LARTC@mailman.ds9a.nl<BR>&gt;&gt;
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
http://lartc.org/<BR>&gt;&gt;<BR>&gt;&gt;<BR>&gt;&gt;<BR>&gt;&gt;
---
 ------------------------------<BR>&gt;&gt; Do You
Yahoo!?<BR>&gt;&gt;
 注册世界一流品质的雅虎免费电邮<BR>&gt;<BR>&gt; ---<BR>&gt;
Catalin(ux aka Dino) BOIE<BR>&gt; catab at
deuroconsult.ro<BR>&gt;
http://kernel.umbrella.ro/<BR>&gt;<BR>&gt;<BR>&gt;
---------------------------------<BR>&gt; Do You
Yahoo!?<BR>&gt;
注册世界一流品质的雅虎免费电邮<BR><BR>---<BR>Catalin(ux aka Dino)
BOIE<BR>catab at
deuroconsult.ro<BR>http://kernel.umbrella.ro/</BLOCKQUOTE><p><br><hr
size=1>
<b>Do You Yahoo!?</b><br>
<a
href="http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/"
target=blank>注册世界一流品质的雅虎免费电邮</a>
--0-1665266126-1106876660=:42398--
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Catalin(ux aka Dino) BOIE

2005-Jan-28 08:16 UTC

head link

Re: Ssh flow does not go into correct class. Help!

CcM

2005-Jan-30 07:26 UTC

head link

Re: Ssh flow does not go into correct class. Help!

You''re right. Thank you very much! 

"Catalin(ux aka Dino) BOIE" <util@deuroconsult.ro> wrote:>
iptables -t mangle -A OUTPUT -d 10.190.6.16 -j MARK --set-mark
20> iptables -t mangle -A OUTPUT -d 10.190.6.16 -j RETURN
>
> tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw flowid
1:11
>
> I expect data from ssh server to my client 10.190.6.16 should goes into 
> 1:11, and other data goes into default 1:13. But when I use ssh client 
> on other machine(not 10.190.6.16), the data seems also goes into 1:11(I 
> got this from tc -s -d class show dev eth0). Why?
Because you didn''t flush the mangle table. Your example works ok for
me.
Add a iptables -t mangle -F before running the script.
>
> "Catalin(ux aka Dino) BOIE" wrote:
> On Thu, 27 Jan 2005, CcM wrote:
>
>> Now I want to control ports with ips specified, like following, it
seems not work, only --sport effect. Ftp-data goes to ips beside 10.190.6.16
also goes into 1:12 while default is 1:13. Why?
>>
>> tc qdisc add dev eth0 root handle 1: htb default 13
>> tc class add dev eth0 parent 1: classid 1:1 htb rate 80000kbit ceil
80000kbit prio 1
>> tc class add dev eth0 parent 1:1 classid 1:11 htb rate 40kbit ceil
40kbit prio 1
>> tc class add dev eth0 parent 1:1 classid 1:12 htb rate 800kbit ceil
1600kbit prio 1
>> tc class add dev eth0 parent 1:1 classid 1:13 htb rate 70000kbit ceil
80000kbit prio 1
>> tc qdisc add dev eth0 parent 1:11 handle 111: sfq perturb 5
>> tc qdisc add dev eth0 parent 1:12 handle 112: sfq perturb 5
>> tc qdisc add dev eth0 parent 1:13 handle 113: sfq perturb 5
>>
>> iptables -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j MARK
--set-mark 20
>> iptables -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j RETURN
>>
>> iptables -t mangle -A POSTROUTING -d 10.190.6.16 -p tcp --sport
ftp-data -j MARK --set-mark 21
>> iptables -t mangle -A POSTROUTING -d 10.190.6.16 -p tcp --sport
ftp-data -j RETURN
>
> I bet you use passive mode in ftp so you get a weird port, not ftp-data
> (20). ftp is tricky to match. You can use this:
> iptables -t mangle -A POSTROUTING -d x.x.x.x -m helper --helper ftp \
> -j MARK --set-mark 21
> This will mark with 21 all ftp control+data connections.
>
>>
>> tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw
flowid 1:11
>> tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 21 fw
flowid 1:12
>>
>>
>> "Catalin(ux aka Dino) BOIE" wrote:
>> On Wed, 26 Jan 2005, CcM wrote:
>>
>>> It does work! Thanks!
>>> But many articles write like what I did, what''s the
difference between POSTROUTING and PREROUTING. I mean I know these are two hook
point in ip stack, but why should be POSTROUTING here when do mangle?
>>>
>>> "Catalin(ux aka Dino) BOIE" wrote:
>>> Use:
>>> iptables -t mangle -A POSTROUTING -p tcp -m tcp --sport 22 -j MARK
\
>>> --set-mark 20
>>
>> Because local generated packets doesn''t go through PREROUTING.
>> But, best is to use OUTPUT.
>>
>>>
>>> ---
>>> Catalin(ux aka Dino) BOIE
>>> catab at deuroconsult.ro
>>> http://kernel.umbrella.ro/
>>> _______________________________________________
>>> LARTC mailing list / LARTC@mailman.ds9a.nl
>>> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
http://lartc.org/
>>>
>>>
>>>
>>> ---------------------------------
>>> Do You Yahoo!?
>>> 注册世界一流品质的雅虎免费电邮
>>
>> ---
>> Catalin(ux aka Dino) BOIE
>> catab at deuroconsult.ro
>> http://kernel.umbrella.ro/
>>
>>
>> ---------------------------------
>> Do You Yahoo!?
>> 注册世界一流品质的雅虎免费电邮
>
> ---
> Catalin(ux aka Dino) BOIE
> catab at deuroconsult.ro
> http://kernel.umbrella.ro/
>
>
> ---------------------------------
> Do You Yahoo!?
> 注册世界一流品质的雅虎免费电邮
---
Catalin(ux aka Dino) BOIE
catab at deuroconsult.ro
http://kernel.umbrella.ro/



---------------------------------
Do You Yahoo!?
注册世界一流品质的雅虎免费电邮
--0-168428698-1107069982=:63766
Content-Type: text/html; charset=gb2312
Content-Transfer-Encoding: 8bit

<DIV>You''re right. Thank you very much!
<BR><BR><B><I>"Catalin(ux aka Dino) BOIE"
&lt;util@deuroconsult.ro&gt;</I></B> wrote:
<BLOCKQUOTE class=replbq style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px;
BORDER-LEFT: #1010ff 2px solid">&gt; iptables -t mangle -A OUTPUT -d
10.190.6.16 -j MARK --set-mark 20<BR>&gt; iptables -t mangle -A OUTPUT
-d 10.190.6.16 -j RETURN<BR>&gt;<BR>&gt; tc filter add dev
eth0 parent 1:0 prio 1 protocol ip handle 20 fw flowid
1:11<BR>&gt;<BR>&gt; I expect data from ssh server to my
client 10.190.6.16 should goes into <BR>&gt; 1:11, and other data goes
into default 1:13. But when I use ssh client <BR>&gt; on other
machine(not 10.190.6.16), the data seems also goes into 1:11(I
<BR>&gt; got this from tc -s -d class show dev eth0).
Why?<BR><BR>Because you didn''t flush the mangle table. Your
example works ok for me.<BR>Add a iptables -t mangle -F before running the
script.<BR><BR>&gt;<BR>&gt; "Catalin(ux aka Dino)
BOIE" <UTIL@DEUROCONSULT.RO>wrote:<BR>&gt; On Thu, 27 Jan
2005, CcM wrote:<BR>&gt;<B
 R>&gt;&gt; Now I want to control ports with ips specified, like
following, it seems not wo
 rk, only
 --sport effect. Ftp-data goes to ips beside 10.190.6.16 also goes into 1:12
while default is 1:13. Why?<BR>&gt;&gt;<BR>&gt;&gt;
tc qdisc add dev eth0 root handle 1: htb default 13<BR>&gt;&gt; tc
class add dev eth0 parent 1: classid 1:1 htb rate 80000kbit ceil 80000kbit prio
1<BR>&gt;&gt; tc class add dev eth0 parent 1:1 classid 1:11 htb
rate 40kbit ceil 40kbit prio 1<BR>&gt;&gt; tc class add dev eth0
parent 1:1 classid 1:12 htb rate 800kbit ceil 1600kbit prio
1<BR>&gt;&gt; tc class add dev eth0 parent 1:1 classid 1:13 htb
rate 70000kbit ceil 80000kbit prio 1<BR>&gt;&gt; tc qdisc add dev
eth0 parent 1:11 handle 111: sfq perturb 5<BR>&gt;&gt; tc qdisc
add dev eth0 parent 1:12 handle 112: sfq perturb 5<BR>&gt;&gt; tc
qdisc add dev eth0 parent 1:13 handle 113: sfq perturb
5<BR>&gt;&gt;<BR>&gt;&gt; iptables -t mangle -A
OUTPUT -p tcp -m tcp --sport 22 -j MARK --set-mark 20<BR>&gt;&gt;
iptabl
 es -t mangle -A OUTPUT -p tcp -m tcp --sport 22 -j
RETURN<BR>&gt;&gt;<BR>&gt;&gt; iptables
  -t
 mangle -A POSTROUTING -d 10.190.6.16 -p tcp --sport ftp-data -j MARK --set-mark
21<BR>&gt;&gt; iptables -t mangle -A POSTROUTING -d 10.190.6.16 -p
tcp --sport ftp-data -j RETURN<BR>&gt;<BR>&gt; I bet you use
passive mode in ftp so you get a weird port, not ftp-data<BR>&gt;
(20). ftp is tricky to match. You can use this:<BR>&gt; iptables -t
mangle -A POSTROUTING -d x.x.x.x -m helper --helper ftp \<BR>&gt; -j
MARK --set-mark 21<BR>&gt; This will mark with 21 all ftp control+data
connections.<BR>&gt;<BR>&gt;&gt;<BR>&gt;&gt;
tc filter add dev eth0 parent 1:0 prio 1 protocol ip handle 20 fw flowid
1:11<BR>&gt;&gt; tc filter add dev eth0 parent 1:0 prio 1 protocol
ip handle 21 fw flowid
1:12<BR>&gt;&gt;<BR>&gt;&gt;<BR>&gt;&gt;
"Catalin(ux aka Dino) BOIE" wrote:<BR>&gt;&gt; On Wed,
26 Jan 2005, CcM
wrote:<BR>&gt;&gt;<BR>&gt;&gt;&gt; It does work!
Thanks!<BR>&gt;&gt;&gt; But many articles write
  like what I did, what''s the difference between POSTROUTING and
PREROUTING. I mean I know
 these
 are two hook point in ip stack, but why should be POSTROUTING here when do
mangle?<BR>&gt;&gt;&gt;<BR>&gt;&gt;&gt;
"Catalin(ux aka Dino) BOIE" wrote:<BR>&gt;&gt;&gt;
Use:<BR>&gt;&gt;&gt; iptables -t mangle -A POSTROUTING -p tcp
-m tcp --sport 22 -j MARK \<BR>&gt;&gt;&gt; --set-mark
20<BR>&gt;&gt;<BR>&gt;&gt; Because local generated
packets doesn''t go through PREROUTING.<BR>&gt;&gt; But,
best is to use
OUTPUT.<BR>&gt;&gt;<BR>&gt;&gt;&gt;<BR>&gt;&gt;&gt;
---<BR>&gt;&gt;&gt; Catalin(ux aka Dino)
BOIE<BR>&gt;&gt;&gt; catab at
deuroconsult.ro<BR>&gt;&gt;&gt;
http://kernel.umbrella.ro/<BR>&gt;&gt;&gt;
_______________________________________________<BR>&gt;&gt;&gt;
LARTC mailing list / LARTC@mailman.ds9a.nl<BR>&gt;&gt;&gt;
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
http://lartc.org/<BR>&gt;&gt;&gt;<BR>&gt;&gt;&gt;<BR>&gt;&gt;&gt;<BR>&gt;&gt;&gt;
---------------------------------<BR>&
 gt;&gt;&gt; Do You Yahoo!?<BR>&gt;&gt;&gt;
注册世界一流品质的雅虎免费电邮<BR>&gt;&gt;<BR>&
 gt;&gt;
 ---<BR>&gt;&gt; Catalin(ux aka Dino)
BOIE<BR>&gt;&gt; catab at
deuroconsult.ro<BR>&gt;&gt;
http://kernel.umbrella.ro/<BR>&gt;&gt;<BR>&gt;&gt;<BR>&gt;&gt;
---------------------------------<BR>&gt;&gt; Do You
Yahoo!?<BR>&gt;&gt;
注册世界一流品质的雅虎免费电邮<BR>&gt;<BR>&gt; ---<BR>&gt;
Catalin(ux aka Dino) BOIE<BR>&gt; catab at
deuroconsult.ro<BR>&gt;
http://kernel.umbrella.ro/<BR>&gt;<BR>&gt;<BR>&gt;
---------------------------------<BR>&gt; Do You
Yahoo!?<BR>&gt;
注册世界一流品质的雅虎免费电邮<BR><BR>---<BR>Catalin(ux aka Dino)
BOIE<BR>catab at deuroconsult.ro<BR><A
href="http://kernel.umbrella.ro/">http://kernel.umbrella.ro/</A></BLOCKQUOTE></DIV><p><br><hr
size=1>
<b>Do You Yahoo!?</b><br>
<a
href="http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/"
target=blank>注册世界一流品质的雅虎免费电邮</a>
--0-168428698-1107069982=:63766--
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Reasonably Related Threads

Search for more apparently analagous threads

LARTC - Jan 2005 - Ssh flow does not go into correct class. Help!

Ssh flow does not go into correct class. Help!

Ssh flow does not go into correct class. Help!

Re: Ssh flow does not go into correct class. Help!

Re: Ssh flow does not go into correct class. Help!

Re: Ssh flow does not go into correct class. Help!

Re: Ssh flow does not go into correct class. Help!

Re: Ssh flow does not go into correct class. Help!

Re: Ssh flow does not go into correct class. Help!

Re: Ssh flow does not go into correct class. Help!

Re: Ssh flow does not go into correct class. Help!

Re: Ssh flow does not go into correct class. Help!

Reasonably Related Threads