My colo provider provides IP addresses and expects routing much like a T-1 data provider. I get 4-5 "WAN" IP addresses and 32 "PUBLIC ROUTABLE" IP addresses. Their enterprise router expects another router (customer provided) to handle this forwarding. In this role, I currently use a RedHat 7.3 box with 2 NICs, simply with IP forwarding enabled. No special rules or shaping. Straight forward enough, and it''s always worked without a hitch. Behind this router, I have another RedHat box acting as a NAT Firewall which protects my server farm. Now my problem. We''ve recently developed an application that makes outgoing requests to other websites and returns data. I''m noticing a serious lag in the amount of time it takes for this data to return to the server vs. our development environment in the office which uses a much slower internet link. If I test from the production RH7.3 "router", all data is returned extremely fast. If I step back to the NAT firewall, or further back into the server farm, I get serious delays. ICMP does not seem to reflect this problem, I''m assuming because of it''s small packet size. Could MTU size be an issue here? All of my firewalls and routers use the default 1500 MTU size and the network is all 100Mbps up to the OC-48 internet backbone. Am I missing some router configuration? I''ve tried adjusting the MTU size on the router with no change in results. Any suggestions on where to go with this? -Ken _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Friday 25 October 2002 17:38, Ken Price wrote:> My colo provider provides IP addresses and expects routing much like a T-1 > data provider. I get 4-5 "WAN" IP addresses and 32 "PUBLIC ROUTABLE" IP > addresses. Their enterprise router expects another router (customer > provided) to handle this forwarding. In this role, I currently use a > RedHat 7.3 box with 2 NICs, simply with IP forwarding enabled. No special > rules or shaping. Straight forward enough, and it''s always worked without > a hitch. Behind this router, I have another RedHat box acting as a NAT > Firewall which protects my server farm. Now my problem. We''ve recently > developed an application that makes outgoing requests to other websites and > returns data. I''m noticing a serious lag in the amount of time it takes for > this data to return to the server vs. our development environment in the > office which uses a much slower internet link. > > If I test from the production RH7.3 "router", all data is returned > extremely fast. If I step back to the NAT firewall, or further back into > the server farm, I get serious delays. ICMP does not seem to reflect this > problem, I''m assuming because of it''s small packet size. Could MTU size be > an issue here? All of my firewalls and routers use the default 1500 MTU > size and the network is all 100Mbps up to the OC-48 internet backbone. Am > I missing some router configuration? I''ve tried adjusting the MTU size on > the router with no change in results. > > Any suggestions on where to go with this?If I have a tcp delay, I always check the dns config. In many cases, there is no reverse dns lookup of the ip-address so the other hosts waits for the dns-timeout before allowing the connection. So, has your ip-address a reverse dns entry? Stef -- stef.coene@docum.org "Using Linux as bandwidth manager" http://www.docum.org/ #lartc @ irc.oftc.net _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
All of our public IPs are reverse mapped. The initial connection to the site is fast. The delay happens when data starts comming back. A way to visualize this problem is using a browser. You hit "Go" and the target site immediately returns text, but like a low-bandwidth or overloaded site, graphics trickle back. This problem is not limited to a single site ... it''s all of them. And isn''t limited to a single router, I have two different production evironments setup with different loadbalancer/firewall combos. What they both have in common is the RedHat router doing simple forwarding. One in each environments. -Ken>> Any suggestions on where to go with this? >If I have a tcp delay, I always check the dns config. In many cases, thereis>no reverse dns lookup of the ip-address so the other hosts waits for the >dns-timeout before allowing the connection. So, has your ip-address a >reverse dns entry?>Stef_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Friday 25 October 2002 18:53, Ken Price wrote:> All of our public IPs are reverse mapped. The initial connection to the > site is fast. The delay happens when data starts comming back. A way to > visualize this problem is using a browser. You hit "Go" and the target > site immediately returns text, but like a low-bandwidth or overloaded site, > graphics trickle back. This problem is not limited to a single site ... > it''s all of them. And isn''t limited to a single router, I have two > different production evironments setup with different loadbalancer/firewall > combos. What they both have in common is the RedHat router doing simple > forwarding. One in each environments.Stange. Have you tried to dump the packets with tcpdump so you can analyse what happens ? Stef -- stef.coene@docum.org "Using Linux as bandwidth manager" http://www.docum.org/ #lartc @ irc.oftc.net _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Yes. Nothing out of the ordinary. I''m seeing packets being sent at 1460 and returned at 1448 - both under my interface''s MTU of 1500. I even tried with MTU sizes down to 500 at the firewall. No luck. The TCPDUMP actually looks just like our development firewall''s TCPDUMP in the office - and development works fine. So I''m lost. The routers in question are Dell 350''s ... PIII-850s with 256Mb RAM with Intel Pro100 NICS, plus one has a 4-port Znyx card (tulip drivers). Page data (just the HTML text) is returned within 1-2 seconds in development or on an outer production router, 8-10 seconds within production. That''s a considerable difference. Leaving firewalling out of it, if you were to setup a Linux router to simply bridge two subnets, after assigning the correct IPs to the interfaces, setting the default gateway to the enterprise router of the ISP, and # echo 1 > /proc/sys/net/ipv4/ip_forward what needs to be done? Ken -----Original Message----- From: Stef Coene [mailto:stef.coene@docum.org] Sent: Friday, October 25, 2002 1:22 PM To: Ken Price; lartc@mailman.ds9a.nl Subject: Re: [LARTC] MTU problem with simple router? On Friday 25 October 2002 18:53, Ken Price wrote:> All of our public IPs are reverse mapped. The initial connection to the > site is fast. The delay happens when data starts comming back. A way to > visualize this problem is using a browser. You hit "Go" and the target > site immediately returns text, but like a low-bandwidth or overloadedsite,> graphics trickle back. This problem is not limited to a single site ... > it''s all of them. And isn''t limited to a single router, I have two > different production evironments setup with differentloadbalancer/firewall> combos. What they both have in common is the RedHat router doing simple > forwarding. One in each environments.Stange. Have you tried to dump the packets with tcpdump so you can analyse what happens ? Stef -- stef.coene@docum.org "Using Linux as bandwidth manager" http://www.docum.org/ #lartc @ irc.oftc.net _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Ok everyone. I want to apologize for this post. The culprit was a poorly configured Cisco 2950 switch (2 of them to be precise). On VLAN creation, STP (spanning tree) is enabled by default. Disabling this feature completely eliminated all the funky latency issues I''ve been experiencing. Doh! -Ken -----Original Message----- From: Ken Price [mailto:kprice@agentware.net] Sent: Friday, October 25, 2002 2:27 PM To: ''Stef Coene''; lartc@mailman.ds9a.nl Subject: RE: [LARTC] MTU problem with simple router? Yes. Nothing out of the ordinary. I''m seeing packets being sent at 1460 and returned at 1448 - both under my interface''s MTU of 1500. I even tried with MTU sizes down to 500 at the firewall. No luck. The TCPDUMP actually looks just like our development firewall''s TCPDUMP in the office - and development works fine. So I''m lost. The routers in question are Dell 350''s ... PIII-850s with 256Mb RAM with Intel Pro100 NICS, plus one has a 4-port Znyx card (tulip drivers). Page data (just the HTML text) is returned within 1-2 seconds in development or on an outer production router, 8-10 seconds within production. That''s a considerable difference. Leaving firewalling out of it, if you were to setup a Linux router to simply bridge two subnets, after assigning the correct IPs to the interfaces, setting the default gateway to the enterprise router of the ISP, and # echo 1 > /proc/sys/net/ipv4/ip_forward what needs to be done? Ken -----Original Message----- From: Stef Coene [mailto:stef.coene@docum.org] Sent: Friday, October 25, 2002 1:22 PM To: Ken Price; lartc@mailman.ds9a.nl Subject: Re: [LARTC] MTU problem with simple router? On Friday 25 October 2002 18:53, Ken Price wrote:> All of our public IPs are reverse mapped. The initial connection to the > site is fast. The delay happens when data starts comming back. A way to > visualize this problem is using a browser. You hit "Go" and the target > site immediately returns text, but like a low-bandwidth or overloadedsite,> graphics trickle back. This problem is not limited to a single site ... > it''s all of them. And isn''t limited to a single router, I have two > different production evironments setup with differentloadbalancer/firewall> combos. What they both have in common is the RedHat router doing simple > forwarding. One in each environments.Stange. Have you tried to dump the packets with tcpdump so you can analyse what happens ? Stef -- stef.coene@docum.org "Using Linux as bandwidth manager" http://www.docum.org/ #lartc @ irc.oftc.net _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/