Can I force traffic down a specific ISP using classification more reliably than with plain marking? As per my last thread (Multi-ISP+ Traffic Shaping) I am having trouble in multi-ISP setup in forcing specific traffic down one ISP or the other. Ideally with a catchall for any unmarked/unclassified traffic. From the way I read http://shorewall.net/3.0/traffic_shaping.htm using classification should enable me to define the outbond interface as well as the traffic shaping rules that should be applied. Is that the correct interpretation? For instance, if I have: tcdevices: #INTERFACE IN-BANDWITH OUT-BANDWIDTH $NET_IF1 $NET_IF1_IN $NET_IF1_OUT $NET_IF2 $NET_IF2_IN $NET_IF2_OUT tcclasses: #INTERFACE MARK RATE CEIL PRIORITY OPTIONS # First ISP - ADSL2+ with a quota $NET_IF1 10 full full 1 tcp-ack,tos-minimize-delay $NET_IF1 20 6*full/10 9*full/10 2 $NET_IF1 30 3*full/10 6*full/10 3 default # Second ISP - ADSL1 without a quota $NET_IF2 40 full full 4 tcp-ack,tos-minimize-delay $NET_IF2 50 6*full/10 9*full/10 5 default tcrules: #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGTH TOS # PORT PORT # Catchall rules - anything unclassifed gets the unmetered (slow) link 2:150 $ANY_IP # The rest of the rules 2:150 $LAN_NETWORK $ANY_IP udp $GAMES 2:150 $LAN_NETWORK $ANY_IP tcp $GAMES 2:150 $LAN_NETWORK $ANY_IP udp $P2P 2:150 $LAN_NETWORK $ANY_IP tcp $P2P 2:150 $LAN_NETWORK $ANY_IP udp $IM 2:150 $LAN_NETWORK $ANY_IP tcp $IM 2:150 $LAN_NETWORK $ANY_IP tcp $ALLMAIL 2:140 $LAN_NETWORK $ANY_IP tcp $STREAM 1:130 $LAN_NETWORK $ANY_IP tcp nntp 1:130 $DMZ1_NETWORK $ANY_IP tcp $ALLMAIL 1:120 $LAN_NETWORK $ANY_IP tcp $FTP 1:120 $LAN_NETWORK $ANY_IP tcp $WWW 1:120 $LAN_NETWORK $ANY_IP tcp ssh - - - 513: 1:110 $LAN_NETWORK $ANY_IP tcp domain 1:110 $LAN_NETWORK $ANY_IP udp domain 1:110 $LAN_NETWORK $ANY_IP tcp ssh - - - 0:512 Is that a better way of over-riding (forcing) the routing? I know the classification step occurs in the POSTROUTING chain which makes me doubt the success of this approach (above). However, I can''t seem to implement BOTH multi-ISP routing *AND* traffic shaping. In my last thread, Jerry Vonau suggested using the interface mark in the PREROUTING chain to send the traffic down one ISP or the other. This worked, but didn''t implement traffic shaping :( Consequently, our SSH sessions all went to hell :-/ So if Jerry was 1/2 right, then maybe I need to specify the shaping in a second set tcrules like this (using the same tcclasses and tcdevices above): # Force ssh down ISP1 1:P $LAN_NETWORK $ANY_IP tcp ssh ... # Make sure ssh has prority on ISP1 10 $LAN_NETWORK $ANY_IP tcp ssh Any further insight would be very much appreciated. Cheers, James ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
On Fri, 2007-08-17 at 13:30 +1000, James Gray wrote:> Can I force traffic down a specific ISP using classification more > reliably than with plain marking?Classification has absolutely nothing to do with ISP selection. It rather selects a class for traffic shaping but the traffic must be going to that interface already (as a result of marking/routing). -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Tom Eastep wrote:> On Fri, 2007-08-17 at 13:30 +1000, James Gray wrote: >> Can I force traffic down a specific ISP using classification more >> reliably than with plain marking? > > Classification has absolutely nothing to do with ISP selection. It > rather selects a class for traffic shaping but the traffic must be going > to that interface already (as a result of marking/routing).Yep - that''s what I thought too (classification happens in POSTROUTING so the route selection has already been made). So I am still stuck with the original problem: I can''t get specific traffic (layer 4) to be routed down a specific ISP. Marking didn''t work (in the FORWARD chain), and marking a packet with the provider mark in PREROUTING seems to bypass the traffic shaping. So three questions: 1. If I mark a packet with the provider''s mark, rather than the mark in tcclasses, what happens (specifically with regard to shaping)? 2. What is the purpose of having a tcclasses mark associate with an interface, if you can''t guarantee the packets with that mark go out the specified interface? (Or can you?) 3. If I can''t force traffic down an particular ISP with a tcrule mark or a provider mark, should I be doing this in the route_rules? (But route_rules doesn''t provide for layer 4 matching). Maybe I''ve been looking at this too long :( Thanks in advance, James ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
On Sat, 2007-08-18 at 14:59 +1000, James Gray wrote:> Tom Eastep wrote: > > On Fri, 2007-08-17 at 13:30 +1000, James Gray wrote: > >> Can I force traffic down a specific ISP using classification more > >> reliably than with plain marking? > > > > Classification has absolutely nothing to do with ISP selection. It > > rather selects a class for traffic shaping but the traffic must be going > > to that interface already (as a result of marking/routing). > > Yep - that''s what I thought too (classification happens in POSTROUTING > so the route selection has already been made). So I am still stuck with > the original problem: I can''t get specific traffic (layer 4) to be > routed down a specific ISP. Marking didn''t work (in the FORWARD chain), > and marking a packet with the provider mark in PREROUTING seems to > bypass the traffic shaping. > > So three questions: > > 1. If I mark a packet with the provider''s mark, rather than the mark in > tcclasses, what happens (specifically with regard to shaping)?If the provider mark happens to be the same as one of the tcclass marks on the interface, then the traffic will be shaped by that class. Otherwise, it will fall into the default class. I implemented HIGH_ROUTE_MARKS=Yes to make the two independent. With that setting, marks used for route selection have values > 255 while marks used for tcclass selection have values <= 255. With HIGH_ROUTE_MARKS=Yes, the mark value is cleared before your FILTER rules so the PREROUTING mark has no effect on tcclass selection.> > 2. What is the purpose of having a tcclasses mark associate with an > interface, if you can''t guarantee the packets with that mark go out the > specified interface? (Or can you?)You don''t seem to be grasping the notion that you must mark the same packet multiple times. You mark it in PREROUTING for interface selection then you re-mark it in FILTER (or use CLASSIFY in POSTROUTING) to specify which tcclass it will be shaped in.> > 3. If I can''t force traffic down an particular ISP with a tcrule mark or > a provider mark, should I be doing this in the route_rules? (But > route_rules doesn''t provide for layer 4 matching).You force traffic to use a particular ISP with PREROUTING marks. You select which tcclass it falls into with FILTER marks or CLASSIFY rules.> > Maybe I''ve been looking at this too long :( >Have you read http://www.shorewall.net/PacketMarking.html ? It might help. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
On Sat, 2007-08-18 at 08:49 -0700, Tom Eastep wrote:> O > > You don''t seem to be grasping the notion that you must mark the same > packet multiple times. You mark it in PREROUTING for interface selection > then you re-mark it in FILTER (or use CLASSIFY in POSTROUTING) to > specify which tcclass it will be shaped in.And you have separate FILTER/CLASSIFY tcrules for each provider -- the outgoing interface name is in the ''DEST'' column. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Tom Eastep wrote:> On Sat, 2007-08-18 at 14:59 +1000, James Gray wrote: >> Tom Eastep wrote: >>> On Fri, 2007-08-17 at 13:30 +1000, James Gray wrote: >>>> Can I force traffic down a specific ISP using classification more >>>> reliably than with plain marking? >>> Classification has absolutely nothing to do with ISP selection. It >>> rather selects a class for traffic shaping but the traffic must be going >>> to that interface already (as a result of marking/routing). >> Yep - that''s what I thought too (classification happens in POSTROUTING >> so the route selection has already been made). So I am still stuck with >> the original problem: I can''t get specific traffic (layer 4) to be >> routed down a specific ISP. Marking didn''t work (in the FORWARD chain), >> and marking a packet with the provider mark in PREROUTING seems to >> bypass the traffic shaping. >> >> So three questions: >> >> 1. If I mark a packet with the provider''s mark, rather than the mark in >> tcclasses, what happens (specifically with regard to shaping)? > > If the provider mark happens to be the same as one of the tcclass marks > on the interface, then the traffic will be shaped by that class. > Otherwise, it will fall into the default class.Agreed.> I implemented HIGH_ROUTE_MARKS=Yes to make the two independent. With > that setting, marks used for route selection have values > 255 while > marks used for tcclass selection have values <= 255. With > HIGH_ROUTE_MARKS=Yes, the mark value is cleared before your FILTER rules > so the PREROUTING mark has no effect on tcclass selection.So in order to use classification marks (eg, 1:150) I need to say "Yes" to HIGH_ROUTE_MARKS *and* specify a mark > 255 in the providers file for that ISP. What if I wanted to specify ISP outbound as follows: providers: #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY $ISP1 1 1 main $NET_IF1 $NET_IF1_GW $PROVOPTS $COPY $ISP2 2 2 main $NET_IF2 $NET_IF2_GW $PROVOPTS $COPY tcrules (abridged): (assuming the same tcclasses I posted earlier) #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGTH TOS # PORT PORT # Route default traffic to ISP2 2:P $ANY_IP # Route traffic to ISP1 1:P $LAN_NETWORK $ANY_IP tcp 22 1:P $LAN_NETWORK $ANY_IP tcp 80,443 # Shape ISP2 Traffic 50 $ANY_IP $ANY_IP udp $GAMES # Shape ISP1 Traffic 20 $LAN_NETWORK $ANY_IP tcp 80,443 10 $LAN_NETWORK $ANY_IP tcp 22>> 2. What is the purpose of having a tcclasses mark associate with an >> interface, if you can''t guarantee the packets with that mark go out the >> specified interface? (Or can you?) > > You don''t seem to be grasping the notion that you must mark the same > packet multiple times. You mark it in PREROUTING for interface selection > then you re-mark it in FILTER (or use CLASSIFY in POSTROUTING) to > specify which tcclass it will be shaped in.I understand the concept, in fact I''ve asked this question 3 times now (in different forms) over the last 2 weeks and this is the first time someone has explicitly stated what I thought from the beginning; a packet has to be marked for an ISP, then re-marked for shaping. Thanks for clearing it up. :) The file syntax seemed to contradict my understanding - you''ve made it too easy :P>> 3. If I can''t force traffic down an particular ISP with a tcrule mark or >> a provider mark, should I be doing this in the route_rules? (But >> route_rules doesn''t provide for layer 4 matching). > > You force traffic to use a particular ISP with PREROUTING marks. You > select which tcclass it falls into with FILTER marks or CLASSIFY rules.The confusion arose from the documentation seeming to suggest routing and shaping could be done on the same line in the tcrules file for any given traffic type and tc class. The reality is, as I suspected, you need to specify route marks in PREROUTING, then shape packets with other marks, BUT they are both done in the tcrules file. Gotcha. Maybe it''s worth explicitly stating this somewhere (MultiISP docs perhaps)? Just a thought - I''m happy to contribute if you like.>> Maybe I''ve been looking at this too long :( >> > > Have you read http://www.shorewall.net/PacketMarking.html ? It might > help.Yep - read that...and re-read it. Makes more sense now that the multiple-marking question has been answered. :) Thanks again. Cheers, James ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
On Sun, 2007-08-19 at 15:49 +1000, James Gray wrote:> > So in order to use classification marks (eg, 1:150) I need to say "Yes" > to HIGH_ROUTE_MARKS *and* specify a mark > 255 in the providers file for > that ISP. What if I wanted to specify ISP outbound as follows: > > providers: > #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY > $ISP1 1 1 main $NET_IF1 $NET_IF1_GW $PROVOPTS $COPY > $ISP2 2 2 main $NET_IF2 $NET_IF2_GW $PROVOPTS $COPY > > tcrules (abridged): > (assuming the same tcclasses I posted earlier) > #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGTH TOS > # PORT PORT > # Route default traffic to ISP2 > 2:P $ANY_IP > > # Route traffic to ISP1 > 1:P $LAN_NETWORK $ANY_IP tcp 22 > 1:P $LAN_NETWORK $ANY_IP tcp 80,443 > > # Shape ISP2 Traffic > 50 $ANY_IP $ANY_IP udp $GAMES > > # Shape ISP1 Traffic > 20 $LAN_NETWORK $ANY_IP tcp 80,443 > 10 $LAN_NETWORK $ANY_IP tcp 22You can do that with two restrictions. a) Shorewall won''t generate rules to clear packet marks at the beginning of the FILTER chains. So you would have to supply two additional rules: 0 0.0.0.0/0 0.0.0.0/0 0 $FW 0.0.0.0/0 b) You won''t be able to use connection marks.> > The confusion arose from the documentation seeming to suggest routing > and shaping could be done on the same line in the tcrules file for any > given traffic type and tc class. The reality is, as I suspected, you > need to specify route marks in PREROUTING, then shape packets with other > marks, BUT they are both done in the tcrules file. Gotcha. Maybe it''s > worth explicitly stating this somewhere (MultiISP docs perhaps)? Just a > thought - I''m happy to contribute if you like.Please do -- the XML source is in the SVN repository (trunk/docs). Cheers, -Tom --> > Tom Eastep \ Nothing is foolproof to a sufficiently talented fool > Shoreline, \ http://shorewall.net > Washington USA \ teastep@shorewall.net > PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Tom Eastep wrote:> On Sun, 2007-08-19 at 15:49 +1000, James Gray wrote: > >> So in order to use classification marks (eg, 1:150) I need to say "Yes" >> to HIGH_ROUTE_MARKS *and* specify a mark > 255 in the providers file for >> that ISP. What if I wanted to specify ISP outbound as follows: >> >> providers: >> #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY >> $ISP1 1 1 main $NET_IF1 $NET_IF1_GW $PROVOPTS $COPY >> $ISP2 2 2 main $NET_IF2 $NET_IF2_GW $PROVOPTS $COPY >> >> tcrules (abridged): >> (assuming the same tcclasses I posted earlier) >> #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGTH TOS >> # PORT PORT >> # Route default traffic to ISP2 >> 2:P $ANY_IP >> >> # Route traffic to ISP1 >> 1:P $LAN_NETWORK $ANY_IP tcp 22 >> 1:P $LAN_NETWORK $ANY_IP tcp 80,443 >> >> # Shape ISP2 Traffic >> 50 $ANY_IP $ANY_IP udp $GAMES >> >> # Shape ISP1 Traffic >> 20 $LAN_NETWORK $ANY_IP tcp 80,443 >> 10 $LAN_NETWORK $ANY_IP tcp 22 > > You can do that with two restrictions. > > a) Shorewall won''t generate rules to clear packet marks at the beginning > of the FILTER chains. So you would have to supply two additional rules: > > 0 0.0.0.0/0 0.0.0.0/0 > 0 $FW 0.0.0.0/0 > > b) You won''t be able to use connection marks.Ok, that makes sense - and explains why you separated the routing/filtering with the HIGH_ROUTE_MARKS options. Now, suppose I replace my provider marks with 0x100 and 0x200, instead of 1 and 2 respectively (and set HIGH_ROUTE_MARKS=Yes), the tcrules file would look the same except for the first 3 PREROUTING rules right? Like this: #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGTH TOS # PORT PORT # Route default traffic to ISP2 0x200:P $ANY_IP # Route traffic to ISP1 0x100:P $LAN_NETWORK $ANY_IP tcp 22 0x100:P $LAN_NETWORK $ANY_IP tcp 80,443>> The confusion arose from the documentation seeming to suggest routing >> and shaping could be done on the same line in the tcrules file for any >> given traffic type and tc class. The reality is, as I suspected, you >> need to specify route marks in PREROUTING, then shape packets with other >> marks, BUT they are both done in the tcrules file. Gotcha. Maybe it''s >> worth explicitly stating this somewhere (MultiISP docs perhaps)? Just a >> thought - I''m happy to contribute if you like. > > Please do -- the XML source is in the SVN repository (trunk/docs).No problem. Once I have this in and working, I''ll put some "put flange ''A'' into slot ''B'' type instructions" ;) Thanks, James ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
James Gray wrote:> > Ok, that makes sense - and explains why you separated the > routing/filtering with the HIGH_ROUTE_MARKS options. Now, suppose I > replace my provider marks with 0x100 and 0x200, instead of 1 and 2 > respectively (and set HIGH_ROUTE_MARKS=Yes), the tcrules file would > look the same except for the first 3 PREROUTING rules right? Like this: > > #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGTH TOS > # PORT PORT > # Route default traffic to ISP2 > 0x200:P $ANY_IP > > # Route traffic to ISP1 > 0x100:P $LAN_NETWORK $ANY_IP tcp 22 > 0x100:P $LAN_NETWORK $ANY_IP tcp 80,443Yes.> > No problem. Once I have this in and working, I''ll put some "put flange > ''A'' into slot ''B'' type instructions" ;) >Great. Thanks, -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/