Neil Greatorex
2004-Sep-23 22:09 UTC
Prioritizing forwarded traffic over locally generated traffic
Hi, I''m a complete newbie at this traffic shaping / QoS stuff so please excuse me if this is a silly question. I''ve searched and searched on Google and I just end up confusing myself even more, so I thought I''d post my question to this list and see whether someone can help me! Basically, I am running a Linux box as a NAT router on my home network (machine name marvin). I want to use mldonkey on the router box for P2P downloads. What I wish to do, is to have any traffic that originates on the internal LAN take priority over traffic that is generated from mldonkey on marvin. I don''t wish to restrict the maximum bandwidth for the P2P downloads on a permanent basis if I can help it - so that all the bandwidth is used all of the time. My plan was to use the PREROUTING and OUTPUT chains of the mangle table to mark the packets, and then use some form of qdisc/class structure that will prioritise one over the other. The aim of this is to have an upload that would normally take say 20 seconds from a machine on the LAN still take 20 seconds when mldonkey is uploading - so the NAT traffic will take all the bandwidth away from mldonkey. The closer to this aim I can get the better! To test implementations, I am using SFTP to upload a file from both a machine on my internal network (named slartibartfast), and marvin (the router machine) simultaneously. The perfect behaviour would be for the upload on slartibartfast to take 20 seconds, and the upload on marvin to take 40. I have tried various setups of qdiscs and classes, using various examples from all over the web (including the LARTC FAQ/cookbook) but I haven''t been able to get anywhere near my aim. All of the attempts I''ve made have led to both uploads taking near enough 40 seconds, as they are both running at 50% of the available bandwidth. I would like it to give almost all the bandwidth to slartibartfast for the first 20 seconds, and then all the bandwidth to marvin for the remaining time. I would really appreciate it if someone could tell me whether: a) This setup is actually possible! b) If using the mangle table chains is correct for this c) If it is, the easiest/best/fastest way to implement it. Just some hints for the right direction would be fine! Many thanks in advance, Neil Greatorex _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Jason Boxman
2004-Sep-23 22:34 UTC
Re: Prioritizing forwarded traffic over locally generated traffic
On Thursday 23 September 2004 18:09, Neil Greatorex wrote:> Hi, > > I''m a complete newbie at this traffic shaping / QoS stuff so please excuse > me if this is a silly question. I''ve searched and searched on Google and I > just end up confusing myself even more, so I thought I''d post my question > to this list and see whether someone can help me!Sure.> Basically, I am running a Linux box as a NAT router on my home network > (machine name marvin). I want to use mldonkey on the router box for P2P > downloads. What I wish to do, is to have any traffic that originates on the > internal LAN take priority over traffic that is generated from mldonkey on > marvin. I don''t wish to restrict the maximum bandwidth for the P2P > downloads on a permanent basis if I can help it - so that all the bandwidth > is used all of the time.So you''d like to classify p2p traffic from mldonkey (Overnet/Kad/eDonkey) such that it is granted a lower priority than other traffic? Not a problem. However, because those three protocols use random ports, you cannot classify ''edonkey'' traffic based on port. You can use either ipp2p or L7-Filter to match these flows based on layer 7 pattern matching, though.> My plan was to use the PREROUTING and OUTPUT chains of the mangle table to > mark the packets, and then use some form of qdisc/class structure that will > prioritise one over the other.I believe you can use the POSTROUTING chain of the mangle table and nab all traffic. L7-Filter has a nice graphic[1] available. [1] http://l7-filter.sourceforge.net/PacketFlow.png> The aim of this is to have an upload that would normally take say 20 > seconds from a machine on the LAN still take 20 seconds when mldonkey is > uploading - so the NAT traffic will take all the bandwidth away from > mldonkey. The closer to this aim I can get the better!That makes sense, although the time interval is relative to the data size and protocol being used, so it isn''t a useful measure for the rest of us. What''s the link size? What''s the file / data size?> To test implementations, I am using SFTP to upload a file from both a > machine on my internal network (named slartibartfast), and marvin (the > router machine) simultaneously. The perfect behaviour would be for the > upload on slartibartfast to take 20 seconds, and the upload on marvin to > take 40.Which implementations have you tried to use? I''d imagine Wondershaper? Others?> I have tried various setups of qdiscs and classes, using various examples > from all over the web (including the LARTC FAQ/cookbook) but I haven''t been > able to get anywhere near my aim. All of the attempts I''ve made have led to > both uploads taking near enough 40 seconds, as they are both running at 50% > of the available bandwidth. I would like it to give almost all the > bandwidth to slartibartfast for the first 20 seconds, and then all the > bandwidth to marvin for the remaining time.The problem is likely that you cannot effectively match p2p flows that use the ''edonkey'' protocols. (Actually, the latest L7-Filter pattern matches do not yet match eMule''s new Kad network, so you''ll still need to either disable support for that in mldonkey or deal with latency issues that arise.)> I would really appreciate it if someone could tell me whether: > a) This setup is actually possible!Absolutely!> b) If using the mangle table chains is correct for thisI believe so.> c) If it is, the easiest/best/fastest way to implement it. Just some hints > for the right direction would be fine!You might explore my guide[2]. I have a setup quite similar to the one you wish to implement, except on my router does not generate any traffic. (I have mldonkey running on an internal machine instead.) [2] http://trekweb.com/~jasonb/articles/traffic_shaping/> Many thanks in advance, > Neil Greatorex >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
kraquen
2004-Sep-24 05:35 UTC
Re: Prioritizing forwarded traffic over locally generated traffic
Sounds to me like he''s trying to match via source IP.. which would catch everything just fine.. Niel, I do something very similar, its fairly simple.. you want to mark packets in your prerouting, then match against them in your qdiscs.. i use an htb.. my upload link can handle about 85 kilobytes / sec. I have several classes that match with various rates, the cieling for all of them is ~80 Then i have a class that matches the mark that i use for that specific IP. That mark goes into a class with a rate of 2 KB/s and a cieling of 75 that class gets 75 when nothing else is running, and 2 if other classes are filling it up. Hope this helps, Jason Jason Boxman wrote:>On Thursday 23 September 2004 18:09, Neil Greatorex wrote: > > >>Hi, >> >>I''m a complete newbie at this traffic shaping / QoS stuff so please excuse >>me if this is a silly question. I''ve searched and searched on Google and I >>just end up confusing myself even more, so I thought I''d post my question >>to this list and see whether someone can help me! >> >> > >Sure. > > > >>Basically, I am running a Linux box as a NAT router on my home network >>(machine name marvin). I want to use mldonkey on the router box for P2P >>downloads. What I wish to do, is to have any traffic that originates on the >>internal LAN take priority over traffic that is generated from mldonkey on >>marvin. I don''t wish to restrict the maximum bandwidth for the P2P >>downloads on a permanent basis if I can help it - so that all the bandwidth >>is used all of the time. >> >> > >So you''d like to classify p2p traffic from mldonkey (Overnet/Kad/eDonkey) such >that it is granted a lower priority than other traffic? Not a problem. >However, because those three protocols use random ports, you cannot classify >''edonkey'' traffic based on port. You can use either ipp2p or L7-Filter to >match these flows based on layer 7 pattern matching, though. > > > >>My plan was to use the PREROUTING and OUTPUT chains of the mangle table to >>mark the packets, and then use some form of qdisc/class structure that will >>prioritise one over the other. >> >> > >I believe you can use the POSTROUTING chain of the mangle table and nab all >traffic. L7-Filter has a nice graphic[1] available. > >[1] http://l7-filter.sourceforge.net/PacketFlow.png > > > >>The aim of this is to have an upload that would normally take say 20 >>seconds from a machine on the LAN still take 20 seconds when mldonkey is >>uploading - so the NAT traffic will take all the bandwidth away from >>mldonkey. The closer to this aim I can get the better! >> >> > >That makes sense, although the time interval is relative to the data size and >protocol being used, so it isn''t a useful measure for the rest of us. What''s >the link size? What''s the file / data size? > > > >>To test implementations, I am using SFTP to upload a file from both a >>machine on my internal network (named slartibartfast), and marvin (the >>router machine) simultaneously. The perfect behaviour would be for the >>upload on slartibartfast to take 20 seconds, and the upload on marvin to >>take 40. >> >> > >Which implementations have you tried to use? I''d imagine Wondershaper? >Others? > > > >>I have tried various setups of qdiscs and classes, using various examples >>from all over the web (including the LARTC FAQ/cookbook) but I haven''t been >>able to get anywhere near my aim. All of the attempts I''ve made have led to >>both uploads taking near enough 40 seconds, as they are both running at 50% >>of the available bandwidth. I would like it to give almost all the >>bandwidth to slartibartfast for the first 20 seconds, and then all the >>bandwidth to marvin for the remaining time. >> >> > >The problem is likely that you cannot effectively match p2p flows that use the >''edonkey'' protocols. (Actually, the latest L7-Filter pattern matches do not >yet match eMule''s new Kad network, so you''ll still need to either disable >support for that in mldonkey or deal with latency issues that arise.) > > > >>I would really appreciate it if someone could tell me whether: >>a) This setup is actually possible! >> >> > >Absolutely! > > > >>b) If using the mangle table chains is correct for this >> >> > >I believe so. > > > >>c) If it is, the easiest/best/fastest way to implement it. Just some hints >>for the right direction would be fine! >> >> > >You might explore my guide[2]. I have a setup quite similar to the one you >wish to implement, except on my router does not generate any traffic. (I >have mldonkey running on an internal machine instead.) > >[2] http://trekweb.com/~jasonb/articles/traffic_shaping/ > > > >>Many thanks in advance, >>Neil Greatorex >> >> >> > >_______________________________________________ >LARTC mailing list / LARTC@mailman.ds9a.nl >http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ > >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Neil Greatorex
2004-Sep-24 16:55 UTC
RE: Prioritizing forwarded traffic over locally generated traffic
Many thanks to both of you for your replies. I have managed to get the setup working how I intended now - by using HTB classes/qdiscs. I had tried this approach before as one of many, however what I had failed to do was create the two classes I am filtering the traffic into as subclasses of a parent HTB class that was limited to the rate of the connection. Now it works as I intended! I''m now going to tackle the harder problem of doing it for downloading - I''m off to play with IMQ :-) Again, many thanks for your suggestions/advice! Cheers, Neil -- #include "sig.h" #define NAME "Neil Greatorex" #define E-MAIL "neil@fatboyfat.co.uk" http://www.spreadfirefox.com/?q=affiliates&id=7889&t=58> -----Original Message----- > From: lartc-admin@mailman.ds9a.nl > [mailto:lartc-admin@mailman.ds9a.nl] On Behalf Of kraquen > Sent: 24 September 2004 6:36 AM > To: jasonb@edseek.com > Cc: lartc@mailman.ds9a.nl > Subject: Re: [LARTC] Prioritizing forwarded traffic over > locally generated traffic > > Sounds to me like he''s trying to match via source IP.. which > would catch > everything just fine.. > > Niel, > I do something very similar, its fairly simple.. > > you want to mark packets in your prerouting, then match > against them in > your qdiscs.. > > i use an htb.. my upload link can handle about 85 kilobytes / sec. > > I have several classes that match with various rates, the cieling for > all of them is ~80 > > Then i have a class that matches the mark that i use for that > specific IP. > > That mark goes into a class with a rate of 2 KB/s and a cieling of 75 > > that class gets 75 when nothing else is running, and 2 if > other classes > are filling it up. > > Hope this helps, > Jason > Jason Boxman wrote: > > >On Thursday 23 September 2004 18:09, Neil Greatorex wrote: > > > > > >>Hi, > >> > >>I''m a complete newbie at this traffic shaping / QoS stuff > so please excuse > >>me if this is a silly question. I''ve searched and searched > on Google and I > >>just end up confusing myself even more, so I thought I''d > post my question > >>to this list and see whether someone can help me! > >> > >> > > > >Sure. > > > > > > > >>Basically, I am running a Linux box as a NAT router on my > home network > >>(machine name marvin). I want to use mldonkey on the router > box for P2P > >>downloads. What I wish to do, is to have any traffic that > originates on the > >>internal LAN take priority over traffic that is generated > from mldonkey on > >>marvin. I don''t wish to restrict the maximum bandwidth for the P2P > >>downloads on a permanent basis if I can help it - so that > all the bandwidth > >>is used all of the time. > >> > >> > > > >So you''d like to classify p2p traffic from mldonkey > (Overnet/Kad/eDonkey) such > >that it is granted a lower priority than other traffic? Not > a problem. > >However, because those three protocols use random ports, you > cannot classify > >''edonkey'' traffic based on port. You can use either ipp2p > or L7-Filter to > >match these flows based on layer 7 pattern matching, though. > > > > > > > >>My plan was to use the PREROUTING and OUTPUT chains of the > mangle table to > >>mark the packets, and then use some form of qdisc/class > structure that will > >>prioritise one over the other. > >> > >> > > > >I believe you can use the POSTROUTING chain of the mangle > table and nab all > >traffic. L7-Filter has a nice graphic[1] available. > > > >[1] http://l7-filter.sourceforge.net/PacketFlow.png > > > > > > > >>The aim of this is to have an upload that would normally take say 20 > >>seconds from a machine on the LAN still take 20 seconds > when mldonkey is > >>uploading - so the NAT traffic will take all the bandwidth away from > >>mldonkey. The closer to this aim I can get the better! > >> > >> > > > >That makes sense, although the time interval is relative to > the data size and > >protocol being used, so it isn''t a useful measure for the > rest of us. What''s > >the link size? What''s the file / data size? > > > > > > > >>To test implementations, I am using SFTP to upload a file > from both a > >>machine on my internal network (named slartibartfast), and > marvin (the > >>router machine) simultaneously. The perfect behaviour would > be for the > >>upload on slartibartfast to take 20 seconds, and the upload > on marvin to > >>take 40. > >> > >> > > > >Which implementations have you tried to use? I''d imagine > Wondershaper? > >Others? > > > > > > > >>I have tried various setups of qdiscs and classes, using > various examples > >>from all over the web (including the LARTC FAQ/cookbook) > but I haven''t been > >>able to get anywhere near my aim. All of the attempts I''ve > made have led to > >>both uploads taking near enough 40 seconds, as they are > both running at 50% > >>of the available bandwidth. I would like it to give almost all the > >>bandwidth to slartibartfast for the first 20 seconds, and > then all the > >>bandwidth to marvin for the remaining time. > >> > >> > > > >The problem is likely that you cannot effectively match p2p > flows that use the > >''edonkey'' protocols. (Actually, the latest L7-Filter > pattern matches do not > >yet match eMule''s new Kad network, so you''ll still need to > either disable > >support for that in mldonkey or deal with latency issues that arise.) > > > > > > > >>I would really appreciate it if someone could tell me whether: > >>a) This setup is actually possible! > >> > >> > > > >Absolutely! > > > > > > > >>b) If using the mangle table chains is correct for this > >> > >> > > > >I believe so. > > > > > > > >>c) If it is, the easiest/best/fastest way to implement it. > Just some hints > >>for the right direction would be fine! > >> > >> > > > >You might explore my guide[2]. I have a setup quite similar > to the one you > >wish to implement, except on my router does not generate any > traffic. (I > >have mldonkey running on an internal machine instead.) > > > >[2] http://trekweb.com/~jasonb/articles/traffic_shaping/ > > > > > > > >>Many thanks in advance, > >>Neil Greatorex > >> > >> > >> > > > >_______________________________________________ > >LARTC mailing list / LARTC@mailman.ds9a.nl > >http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: > http://lartc.org/ > > > > > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Jason Boxman
2004-Sep-24 17:07 UTC
Re: Prioritizing forwarded traffic over locally generated traffic
On Friday 24 September 2004 12:55, Neil Greatorex wrote:> Many thanks to both of you for your replies. > > I have managed to get the setup working how I intended now - by using HTB > classes/qdiscs. I had tried this approach before as one of many, however > what I had failed to do was create the two classes I am filtering the > traffic into as subclasses of a parent HTB class that was limited to the > rate of the connection. Now it works as I intended! > > I''m now going to tackle the harder problem of doing it for downloading - > I''m off to play with IMQ :-)Indeed. Some people experience success setting up another egress qdisc on the router''s internet NIC and shaping with that. Since you have lots of ingress to the router itself destined for the router, this may not work. I haven''t messed with IMQ yet, but it ought to work too. It''s just extra work if you don''t need to shape incoming traffic for the router itself as well as the network.> Again, many thanks for your suggestions/advice!Enjoy! _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Alexander Samad
2004-Oct-07 03:50 UTC
Re: Prioritizing forwarded traffic over locally generated traffic
Hi would it be possible to post the scripts that set this up ??? Alex On Fri, Sep 24, 2004 at 05:55:36PM +0100, Neil Greatorex wrote:> Many thanks to both of you for your replies. > > I have managed to get the setup working how I intended now - by using HTB > classes/qdiscs. I had tried this approach before as one of many, however > what I had failed to do was create the two classes I am filtering the > traffic into as subclasses of a parent HTB class that was limited to the > rate of the connection. Now it works as I intended! > > I''m now going to tackle the harder problem of doing it for downloading - I''m > off to play with IMQ :-) > > Again, many thanks for your suggestions/advice! > > Cheers, > Neil > > -- > #include "sig.h" > #define NAME "Neil Greatorex" > #define E-MAIL "neil@fatboyfat.co.uk" > > http://www.spreadfirefox.com/?q=affiliates&id=7889&t=58 > > > > > -----Original Message----- > > From: lartc-admin@mailman.ds9a.nl > > [mailto:lartc-admin@mailman.ds9a.nl] On Behalf Of kraquen > > Sent: 24 September 2004 6:36 AM > > To: jasonb@edseek.com > > Cc: lartc@mailman.ds9a.nl > > Subject: Re: [LARTC] Prioritizing forwarded traffic over > > locally generated traffic > > > > Sounds to me like he''s trying to match via source IP.. which > > would catch > > everything just fine.. > > > > Niel, > > I do something very similar, its fairly simple.. > > > > you want to mark packets in your prerouting, then match > > against them in > > your qdiscs.. > > > > i use an htb.. my upload link can handle about 85 kilobytes / sec. > > > > I have several classes that match with various rates, the cieling for > > all of them is ~80 > > > > Then i have a class that matches the mark that i use for that > > specific IP. > > > > That mark goes into a class with a rate of 2 KB/s and a cieling of 75 > > > > that class gets 75 when nothing else is running, and 2 if > > other classes > > are filling it up. > > > > Hope this helps, > > Jason > > Jason Boxman wrote: > > > > >On Thursday 23 September 2004 18:09, Neil Greatorex wrote: > > > > > > > > >>Hi, > > >> > > >>I''m a complete newbie at this traffic shaping / QoS stuff > > so please excuse > > >>me if this is a silly question. I''ve searched and searched > > on Google and I > > >>just end up confusing myself even more, so I thought I''d > > post my question > > >>to this list and see whether someone can help me! > > >> > > >> > > > > > >Sure. > > > > > > > > > > > >>Basically, I am running a Linux box as a NAT router on my > > home network > > >>(machine name marvin). I want to use mldonkey on the router > > box for P2P > > >>downloads. What I wish to do, is to have any traffic that > > originates on the > > >>internal LAN take priority over traffic that is generated > > from mldonkey on > > >>marvin. I don''t wish to restrict the maximum bandwidth for the P2P > > >>downloads on a permanent basis if I can help it - so that > > all the bandwidth > > >>is used all of the time. > > >> > > >> > > > > > >So you''d like to classify p2p traffic from mldonkey > > (Overnet/Kad/eDonkey) such > > >that it is granted a lower priority than other traffic? Not > > a problem. > > >However, because those three protocols use random ports, you > > cannot classify > > >''edonkey'' traffic based on port. You can use either ipp2p > > or L7-Filter to > > >match these flows based on layer 7 pattern matching, though. > > > > > > > > > > > >>My plan was to use the PREROUTING and OUTPUT chains of the > > mangle table to > > >>mark the packets, and then use some form of qdisc/class > > structure that will > > >>prioritise one over the other. > > >> > > >> > > > > > >I believe you can use the POSTROUTING chain of the mangle > > table and nab all > > >traffic. L7-Filter has a nice graphic[1] available. > > > > > >[1] http://l7-filter.sourceforge.net/PacketFlow.png > > > > > > > > > > > >>The aim of this is to have an upload that would normally take say 20 > > >>seconds from a machine on the LAN still take 20 seconds > > when mldonkey is > > >>uploading - so the NAT traffic will take all the bandwidth away from > > >>mldonkey. The closer to this aim I can get the better! > > >> > > >> > > > > > >That makes sense, although the time interval is relative to > > the data size and > > >protocol being used, so it isn''t a useful measure for the > > rest of us. What''s > > >the link size? What''s the file / data size? > > > > > > > > > > > >>To test implementations, I am using SFTP to upload a file > > from both a > > >>machine on my internal network (named slartibartfast), and > > marvin (the > > >>router machine) simultaneously. The perfect behaviour would > > be for the > > >>upload on slartibartfast to take 20 seconds, and the upload > > on marvin to > > >>take 40. > > >> > > >> > > > > > >Which implementations have you tried to use? I''d imagine > > Wondershaper? > > >Others? > > > > > > > > > > > >>I have tried various setups of qdiscs and classes, using > > various examples > > >>from all over the web (including the LARTC FAQ/cookbook) > > but I haven''t been > > >>able to get anywhere near my aim. All of the attempts I''ve > > made have led to > > >>both uploads taking near enough 40 seconds, as they are > > both running at 50% > > >>of the available bandwidth. I would like it to give almost all the > > >>bandwidth to slartibartfast for the first 20 seconds, and > > then all the > > >>bandwidth to marvin for the remaining time. > > >> > > >> > > > > > >The problem is likely that you cannot effectively match p2p > > flows that use the > > >''edonkey'' protocols. (Actually, the latest L7-Filter > > pattern matches do not > > >yet match eMule''s new Kad network, so you''ll still need to > > either disable > > >support for that in mldonkey or deal with latency issues that arise.) > > > > > > > > > > > >>I would really appreciate it if someone could tell me whether: > > >>a) This setup is actually possible! > > >> > > >> > > > > > >Absolutely! > > > > > > > > > > > >>b) If using the mangle table chains is correct for this > > >> > > >> > > > > > >I believe so. > > > > > > > > > > > >>c) If it is, the easiest/best/fastest way to implement it. > > Just some hints > > >>for the right direction would be fine! > > >> > > >> > > > > > >You might explore my guide[2]. I have a setup quite similar > > to the one you > > >wish to implement, except on my router does not generate any > > traffic. (I > > >have mldonkey running on an internal machine instead.) > > > > > >[2] http://trekweb.com/~jasonb/articles/traffic_shaping/ > > > > > > > > > > > >>Many thanks in advance, > > >>Neil Greatorex > > >> > > >> > > >> > > > > > >_______________________________________________ > > >LARTC mailing list / LARTC@mailman.ds9a.nl > > >http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: > > http://lartc.org/ > > > > > > > > > > _______________________________________________ > > LARTC mailing list / LARTC@mailman.ds9a.nl > > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ > > > > > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >