Hey guys, I have several Linux routers in place at high-usage locations (student apartment complexes). I''m having trouble with some of the routers which use 6Mbit DSL lines as their Internet feed. The routers use PPPoE and perform NAT. During peak usage periods, the routers are dropping alot of packets. I''m lead to believe this is because there are too many active connections. For example, when I ping the WAN IP address of one of the routers from a remote location, I may start getting replies immediately. But during peak periods, the first several pings usually time out and then they just start responding. Sometimes they start responding on the 4th ping, sometimes the 12th, etc., it''s pretty random. I searched the web and tried increasing my gc_cache settings, but it didn''t make a difference. echo 512 > /proc/sys/net/ipv4/neigh/default/gc_thresh1 echo 2048 > /proc/sys/net/ipv4/neigh/default/gc_thresh2 echo 4096 > /proc/sys/net/ipv4/neigh/default/gc_thresh3 The other notable difference is that the conntrack tables are much larger than normal. `wc -l /proc/net/ip_conntrack` returns >19000 on the routers experiencing packet loss while virtually all of the other routers (not having this issue) have less than 5000 entries in ip_conntrack. I tried increasing ip_conntrack_max in /proc, setting it to 65536 - didn''t make a difference. Are there any other /proc settings I should change to improve performance? Any tips on analyzing the ip_conntrack data to find oddities? FYI I''m using kernel 2.4.25. I''d rather not upgrade to 2.6 since doing so in the past has introduced more problems! Thanks. ____________________________________________________________________________________ No need to miss a message. Get email on-the-go with Yahoo! Mail for Mobile. Get started. http://mobile.yahoo.com/mail
Hi Do you block P2P traffic in your routers?, you might use module ipp2p, How many RAM do you have in your routers Linux?. Assure that MTU is configured for lower 1500 in your networks cards, in many cases 1492. On 3/5/07, John Philips <johnphilips42@yahoo.com> wrote:> Hey guys, > > I have several Linux routers in place at high-usage > locations (student apartment complexes). I''m having > trouble with some of the routers which use 6Mbit DSL > lines as their Internet feed. The routers use PPPoE > and perform NAT. > > During peak usage periods, the routers are dropping > alot of packets. I''m lead to believe this is because > there are too many active connections. > > For example, when I ping the WAN IP address of one of > the routers from a remote location, I may start > getting replies immediately. But during peak periods, > the first several pings usually time out and then they > just start responding. Sometimes they start > responding on the 4th ping, sometimes the 12th, etc., > it''s pretty random. > > I searched the web and tried increasing my gc_cache > settings, but it didn''t make a difference. > > echo 512 > /proc/sys/net/ipv4/neigh/default/gc_thresh1 > echo 2048 > > /proc/sys/net/ipv4/neigh/default/gc_thresh2 > echo 4096 > > /proc/sys/net/ipv4/neigh/default/gc_thresh3 > > The other notable difference is that the conntrack > tables are much larger than normal. > > `wc -l /proc/net/ip_conntrack` returns >19000 on the > routers experiencing packet loss while virtually all > of the other routers (not having this issue) have less > than 5000 entries in ip_conntrack. I tried increasing > ip_conntrack_max in /proc, setting it to 65536 - > didn''t make a difference. > > Are there any other /proc settings I should change to > improve performance? Any tips on analyzing the > ip_conntrack data to find oddities? > > FYI I''m using kernel 2.4.25. I''d rather not upgrade > to 2.6 since doing so in the past has introduced more > problems! > > Thanks. > > > > > > ____________________________________________________________________________________ > No need to miss a message. Get email on-the-go > with Yahoo! Mail for Mobile. Get started. > http://mobile.yahoo.com/mail > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >-- "The network is the computer"
John Philips wrote:> Hey guys, > > I have several Linux routers in place at high-usage > locations (student apartment complexes). I''m having > trouble with some of the routers which use 6Mbit DSL > lines as their Internet feed. The routers use PPPoE > and perform NAT. > > During peak usage periods, the routers are dropping > alot of packets. I''m lead to believe this is because > there are too many active connections.Besides what you wrote in the rest of your mail, do you have any other reason to believe this? Based on the information you''ve given, I would suspect you''re just seeing the normal (albeit ugly) effects of saturating a DSL line. Are your Linux routers doing any traffic shaping? When you''re having these problems, what is the bandwidth going over the DSL? Don''t forget to look at both upstream and downstream rates. -Corey
Guys, I called my DSL provider and it turns out they limit the number of simultaneous "flows" you can have. I guess that means active TCP connections. Their limit is 1500 concurrent flows, and when the tech looked at it we had 1450 active. I presume all these flows are from P2P users, so I''m going to try using the connlimit iptables extension to prevent individual users from having more than 50 or so connections. --- John Philips <johnphilips42@yahoo.com> wrote:> Hey guys, > > I have several Linux routers in place at high-usage > locations (student apartment complexes). I''m having > trouble with some of the routers which use 6Mbit DSL > lines as their Internet feed. The routers use PPPoE > and perform NAT. > > During peak usage periods, the routers are dropping > alot of packets. I''m lead to believe this is > because > there are too many active connections. > > For example, when I ping the WAN IP address of one > of > the routers from a remote location, I may start > getting replies immediately. But during peak > periods, > the first several pings usually time out and then > they > just start responding. Sometimes they start > responding on the 4th ping, sometimes the 12th, > etc., > it''s pretty random. > > I searched the web and tried increasing my gc_cache > settings, but it didn''t make a difference. > > echo 512 > > /proc/sys/net/ipv4/neigh/default/gc_thresh1 > echo 2048 > > /proc/sys/net/ipv4/neigh/default/gc_thresh2 > echo 4096 > > /proc/sys/net/ipv4/neigh/default/gc_thresh3 > > The other notable difference is that the conntrack > tables are much larger than normal. > > `wc -l /proc/net/ip_conntrack` returns >19000 on the > routers experiencing packet loss while virtually all > of the other routers (not having this issue) have > less > than 5000 entries in ip_conntrack. I tried > increasing > ip_conntrack_max in /proc, setting it to 65536 - > didn''t make a difference. > > Are there any other /proc settings I should change > to > improve performance? Any tips on analyzing the > ip_conntrack data to find oddities? > > FYI I''m using kernel 2.4.25. I''d rather not upgrade > to 2.6 since doing so in the past has introduced > more > problems! > > Thanks. > > > > > >____________________________________________________________________________________> No need to miss a message. Get email on-the-go > with Yahoo! Mail for Mobile. Get started. > http://mobile.yahoo.com/mail > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl >http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc>____________________________________________________________________________________ Don''t get soaked. Take a quick peek at the forecast with the Yahoo! Search weather shortcut. http://tools.search.yahoo.com/shortcuts/#loc_weather
John Philips wrote:> Guys, > > I called my DSL provider and it turns out they limit > the number of simultaneous "flows" you can have. I > guess that means active TCP connections.But you saw problems with icmp...., but as Corey said you may have Qos problems aswell Their limit> is 1500 concurrent flows, and when the tech looked at > it we had 1450 active.Ewww that''s horrible - I can eat 2k conntracks all by myself. Andy.