I''ve been doing ipchains/tables firewalls for quite a while now, but I''d like to be able to do some bandwith shaping, and src-address based routing to specific net connections. From what I''ve read so far, tc/ip can do that, correct? Here''s simply what I''ve got: Linux firewall, 2.4.x, iptables, MASQ, about 10 internal subnets all going out as 1 IP now. I have 1 T1 now, 2 more on the way. I''d like to say internal subnets 1-5 go out T1 #1 6-7 go out T1 #2, etc. I''d also like to be able to say that subnets 1-5 can only have 50% of that T1. I know these are 2 seperate questions, but that''s where I would like to be. Can someone point me in the direction of some good docs/examples on how to set this up? What I''m also unclear about, is if I''m using ip rules to reroute traffic to specific out-going routers, do I still need MASQ, and what role does it play? Thanks... Dan
On Wed, Dec 19, 2001 at 02:06:01AM -0600, Daniel Wittenberg wrote:> I''ve been doing ipchains/tables firewalls for quite a while now, but I''d > like to be able to do some bandwith shaping, and src-address based > routing to specific net connections. From what I''ve read so far, tc/ip > can do that, correct? Here''s simply what I''ve got: Linux firewall, > 2.4.x, iptables, MASQ, about 10 internal subnets all going out as 1 IP > now. I have 1 T1 now, 2 more on the way. I''d like to say internal > subnets 1-5 go out T1 #1 6-7 go out T1 #2, etc. I''d also like to be > able to say that subnets 1-5 can only have 50% of that T1. I know these > are 2 seperate questions, but that''s where I would like to be. Can > someone point me in the direction of some good docs/examples on how to > set this up? What I''m also unclear about, is if I''m using ip rules toYou need: - to mark traffic coming in from different subnets - route based on that mark You can achieve the first with iptables in the mangle/PREROUTING target with fwmark. The second can be done with ''ip rule add'' and ''ip route add .. table x''.> reroute traffic to specific out-going routers, do I still need MASQ, and > what role does it play?MASQ leaves the fwmark alone, so it doesn''t interfere. Regards, bert -- http://www.PowerDNS.com Versatile DNS Software & Services http://www.tk the dot in .tk Netherlabs BV / Rent-a-Nerd.nl - Nerd Available - Linux Advanced Routing & Traffic Control: http://ds9a.nl/lartc
> I''ve been doing ipchains/tables firewalls for quite a while now, but I''d > like to be able to do some bandwith shaping, and src-address based > routing to specific net connections.I''ll again do some advertisement for my bytelimit patch :-) it is a patch for netfilter (iptables) allowing to limit bandwidth, like the "limit" match but allowing to specify rates in bytes/second instead of packets/second. you might combine this patch and SNAT to do "overflowing", i.e. your first link will be used, and when it is "full" (or exceeds a given bandwidth), further connections will be SNAT''ed with another address, thus using the 2nd link. Jerome Petazzoni <skaya at enix dot org>
On Wed, Dec 19, 2001 at 07:07:42PM +0100, Jerome PETAZZONI wrote:> > > I''ve been doing ipchains/tables firewalls for quite a while now, but I''d > > like to be able to do some bandwith shaping, and src-address based > > routing to specific net connections. > > I''ll again do some advertisement for my bytelimit patch :-) > it is a patch for netfilter (iptables) allowing to limit bandwidth, > like the "limit" match but allowing to specify rates in bytes/second > instead of packets/second.Does it have a peakrate? If not, why not? regards, bert -- http://www.PowerDNS.com Versatile DNS Software & Services http://www.tk the dot in .tk Netherlabs BV / Rent-a-Nerd.nl - Nerd Available - Linux Advanced Routing & Traffic Control: http://ds9a.nl/lartc
Just to clarify how this works, so I know it''s going to work for what I need: Let''s say I have 10 subnets, I mark the packets coming from 3 of them with a 1. Now I just set an out-going limit on packets marked with 1 to let''s say 100 bytes/second. Now, does the 100 bytes/sec apply to each connection, or to _all_ packets with that mark? What I want to make sure is that the entire subnet is limited to 100 bytes/sec, and not individual connections or src addy''s...? Dan On Wed, 2001-12-19 at 12:07, Jerome PETAZZONI wrote:> > > I''ve been doing ipchains/tables firewalls for quite a while now, but I''d > > like to be able to do some bandwith shaping, and src-address based > > routing to specific net connections. > > I''ll again do some advertisement for my bytelimit patch :-) > it is a patch for netfilter (iptables) allowing to limit bandwidth, > like the "limit" match but allowing to specify rates in bytes/second > instead of packets/second. > > you might combine this patch and SNAT to do "overflowing", i.e. > your first link will be used, and when it is "full" (or exceeds > a given bandwidth), further connections will be SNAT''ed with > another address, thus using the 2nd link. > > Jerome Petazzoni <skaya at enix dot org>
You are talking about marks which have to stay in one machine. It seems more general to have marks that can travel on your network. If you add the RIFRAF Routing changes to FreeBSD, then you can do all sorts of things on a per /16 prefix basis. For example, you can easily mark packets for sorting into flows, or mark them with random tags to have them routed to two links in a load-balancing mode. http://www.dot-biz.com/IPv4/Tutorial/ http://www.dot-biz.com/IPv4/Tutorial/RIFRAFBSD4.4.txt The simple ping command is used to control the RIFRAF settings in another machine, so, in theory, you can reach across the entire global Internet and change the way packets are handled for specific /16 prefixes. 10.0 and 192.168 are commonly used. RIFRAF - Remote Identification Field Random Action Filter Jim Fleming http://www.IPv8.info IPv16....One Better !! ----- Original Message ----- From: "Daniel Wittenberg" <daniel-wittenberg@starken.com> To: <skaya@enix.org> Cc: <lartc@mailman.ds9a.nl> Sent: Wednesday, December 19, 2001 11:46 PM Subject: RE: [LARTC] shaping/routing> Just to clarify how this works, so I know it''s going to work for what I > need: Let''s say I have 10 subnets, I mark the packets coming from 3 of > them with a 1. Now I just set an out-going limit on packets marked with > 1 to let''s say 100 bytes/second. Now, does the 100 bytes/sec apply to > each connection, or to _all_ packets with that mark? What I want to > make sure is that the entire subnet is limited to 100 bytes/sec, and not > individual connections or src addy''s...? > > Dan > > On Wed, 2001-12-19 at 12:07, Jerome PETAZZONI wrote: > > > > > I''ve been doing ipchains/tables firewalls for quite a while now, but I''d > > > like to be able to do some bandwith shaping, and src-address based > > > routing to specific net connections. > > > > I''ll again do some advertisement for my bytelimit patch :-) > > it is a patch for netfilter (iptables) allowing to limit bandwidth, > > like the "limit" match but allowing to specify rates in bytes/second > > instead of packets/second. > > > > you might combine this patch and SNAT to do "overflowing", i.e. > > your first link will be used, and when it is "full" (or exceeds > > a given bandwidth), further connections will be SNAT''ed with > > another address, thus using the 2nd link. > > > > Jerome Petazzoni <skaya at enix dot org> > > > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/ >
Yes, I am only talking about just one firewall machine that is doing all the routing, so I''m not sure how what you have suggest helps. Unless what you are saying is "NO, you can''t do that" to what I asked, and that what I want to do effects 1 machine at a time and not everything that is marked. You didn''t really answer the question, just talk about alternatives, which doesn''t help me understand how the --limit works. Don''t mean to sound ranting, just want to understand so I need to have questions answered before getting other suggestions... Dan On Thu, 2001-12-20 at 00:07, Jim Fleming wrote:> You are talking about marks which have to stay in one machine. > It seems more general to have marks that can travel on your network. >
>> > I''ve been doing ipchains/tables firewalls for quite a while now, but I''d >> > like to be able to do some bandwith shaping, and src-address based >> > routing to specific net connections.>> I''ll again do some advertisement for my bytelimit patch :-)> Just to clarify how this works, so I know it''s going to work for what I > need: Let''s say I have 10 subnets, I mark the packets coming from 3 of > them with a 1. Now I just set an out-going limit on packets marked with > 1 to let''s say 100 bytes/second. Now, does the 100 bytes/sec apply to > each connection, or to _all_ packets with that mark?all packets with that mark. each packet running thru the rule with the bytelimit match will be accounted for the limitation. on the other hand, if you wanted to limit by connection, my patch can''t do it :( but when I''ll be more familiar with conntracking code, I''ll try to do it. Jerome Petazzoni <skaya at enix dot org> -- ''Oh, them as makes the endings don''t get them,'' said Granny. (Maskerade)
>> I''ll again do some advertisement for my bytelimit patch :-) >> it is a patch for netfilter (iptables) allowing to limit bandwidth, >> like the "limit" match but allowing to specify rates in bytes/second >> instead of packets/second.> Does it have a peakrate? If not, why not?sort of... it has a very simple algorithm : each "bytelimit" has a bucket of "tokens", each "token" allowing 1 byte to pass. the bucket has a maximal size, and "gains" X tokens per second, where X is the "nominal rate". you can set separately the bucket maximal size and the rate, so for instance, if you set 1000 bytes/second "rate", and 10000 "bucket size", you''ll be able to do 2000 bytes/second during 10 seconds, or 10000 bytes/second during 1 second, and so on. of course, you can combine two rules, if you want to allow 1000 bytes per second on average, and 2000 bytes per second while 10 seconds but no more, just chain a 1000 bps rule with 10000 bucketsize, and a 2000 bps rule with 1600 bucketsize. the "rule of thumb" for bucketsize calculations should be : - no less than 1600 (that''s roughly one ethernet frame) - rate/HZ for minimal burstiness (IIRC, HZ is 100 for intel, 1024 for alpha, don''t know for others) of course, this patch is not as powerful as the full QoS+tc suite ; but it allows very simple and straightforward shaping. IMHO, the biggest flaw is the lack of qdisc, so it would be interesting to setup a 3-band qdisc respecting TOS marks, and set TOS marks with iptables. and before people start asking why is the point of this patch : if you have a nice way to configure your iptables rules (web interface for customers, or whatever), you are *very happy* to integrate bandwidth shaping into it, instead of having to design another interface for QoS, which would have to be coupled with the first one anyway :( . regards, Jerome Petazzoni <skaya at enix dot org> -- ''Things either exist or they don''t,'' said Jeremy. ''I am very clear about that. I have medicine.'' (The Thief of Time)
Ok, so I think I understand this simple routing question. I have several internal subnets: 172.16.1.0/24, 172.16.2.0/24, etc. I have 2 out-going network links: 1.1.1.1 and 2.2.2.2 So I can change the out-going route by using: echo "100 link1" >> /etc/iproute2/rt_tables ip rule add from 172.16.1.0/24 table link1 ip rule add from 172.16.2.0/24 table link1 ip route add default via 1.1.1.1 dev $EXTERNAL_INTERFACE table link1 echo "200 link2" >> /etc/iproute2/rt_tables ip rule add from 172.16.3.0/24 table link2 ip rule add from 172.16.4.0/24 table link2 ip rule add from 172.16.5.0/24 table link2 ip route add default via 2.2.2.2 dev $EXTERNAL_INTERFACE table link2 and if I understand this correctly, this doesn''t change anything in iptables, so I can still this at the end of my firewall script. I''ll also want to use SNAT and change the out-going IP''s so they come back on the correct link. Look good? I can''t test this in production very often, so before I do I thought I''d run it by everyone since I''m very much a newbie on tc... Thanks! Dan
On Mon, Dec 24, 2001 at 10:58:14PM -0600, Daniel Wittenberg wrote:> So I can change the out-going route by using: > echo "100 link1" >> /etc/iproute2/rt_tables > ip rule add from 172.16.1.0/24 table link1 > ip rule add from 172.16.2.0/24 table link1 > ip route add default via 1.1.1.1 dev $EXTERNAL_INTERFACE table link1 > > echo "200 link2" >> /etc/iproute2/rt_tables > ip rule add from 172.16.3.0/24 table link2 > ip rule add from 172.16.4.0/24 table link2 > ip rule add from 172.16.5.0/24 table link2 > ip route add default via 2.2.2.2 dev $EXTERNAL_INTERFACE table link2Looks good.> and if I understand this correctly, this doesn''t change anything in > iptables, so I can still this at the end of my firewall script. I''ll > also want to use SNAT and change the out-going IP''s so they come back on > the correct link. Look good? I can''t test this in production very > often, so before I do I thought I''d run it by everyone since I''m very > much a newbie on tc...We use a combination of SNAT/DNAT, MASQ, CBQ, iptables AND Policy routing in production, and it all works fine. Regards, bert -- http://www.PowerDNS.com Versatile DNS Software & Services http://www.tk the dot in .tk Netherlabs BV / Rent-a-Nerd.nl - Nerd Available - Linux Advanced Routing & Traffic Control: http://ds9a.nl/lartc