My asterisk server is getting bogged down every 5 minutes. My ping time is
going from 60ms to 800 ms and the call quality is bad.
I have fail2ban running and I am using iptables. I have two ip connections
to the box.
How can I tell if the poor performance is due to sip attacks? I don't see
any reg attempts in my asterisk cli. I use to get frequent attacks but
fail2ban seems to be taking care of that.
See how ping time gets worst in a short space of time and server performance
at the time:
64 bytes from 4.2.2.1: icmp_seq=6 ttl=55 time=87.8 ms
64 bytes from 4.2.2.1: icmp_seq=7 ttl=55 time=99.8 ms
64 bytes from 4.2.2.1: icmp_seq=8 ttl=55 time=107 ms
64 bytes from 4.2.2.1: icmp_seq=9 ttl=55 time=115 ms
64 bytes from 4.2.2.1: icmp_seq=10 ttl=55 time=120 ms
64 bytes from 4.2.2.1: icmp_seq=11 ttl=55 time=122 ms
64 bytes from 4.2.2.1: icmp_seq=12 ttl=55 time=123 ms
64 bytes from 4.2.2.1: icmp_seq=13 ttl=55 time=126 ms
64 bytes from 4.2.2.1: icmp_seq=14 ttl=55 time=122 ms
64 bytes from 4.2.2.1: icmp_seq=15 ttl=55 time=142 ms
64 bytes from 4.2.2.1: icmp_seq=16 ttl=55 time=142 ms
64 bytes from 4.2.2.1: icmp_seq=17 ttl=55 time=137 ms
64 bytes from 4.2.2.1: icmp_seq=18 ttl=55 time=186 ms
64 bytes from 4.2.2.1: icmp_seq=19 ttl=55 time=255 ms
64 bytes from 4.2.2.1: icmp_seq=20 ttl=55 time=310 ms
64 bytes from 4.2.2.1: icmp_seq=21 ttl=55 time=387 ms
64 bytes from 4.2.2.1: icmp_seq=22 ttl=55 time=445 ms
64 bytes from 4.2.2.1: icmp_seq=23 ttl=55 time=514 ms
64 bytes from 4.2.2.1: icmp_seq=24 ttl=55 time=583 ms
64 bytes from 4.2.2.1: icmp_seq=25 ttl=55 time=650 ms
64 bytes from 4.2.2.1: icmp_seq=26 ttl=55 time=715 ms
64 bytes from 4.2.2.1: icmp_seq=27 ttl=55 time=783 ms
64 bytes from 4.2.2.1: icmp_seq=28 ttl=55 time=821 ms
64 bytes from 4.2.2.1: icmp_seq=29 ttl=55 time=810 ms
64 bytes from 4.2.2.1: icmp_seq=30 ttl=55 time=832 ms
64 bytes from 4.2.2.1: icmp_seq=31 ttl=55 time=812 ms
64 bytes from 4.2.2.1: icmp_seq=32 ttl=55 time=821 ms
64 bytes from 4.2.2.1: icmp_seq=33 ttl=55 time=826 ms
64 bytes from 4.2.2.1: icmp_seq=34 ttl=55 time=815 ms
64 bytes from 4.2.2.1: icmp_seq=35 ttl=55 time=821 ms
64 bytes from 4.2.2.1: icmp_seq=36 ttl=55 time=824 ms
top - 19:02:38 up 4 days, 11:26, 4 users, load average: 0.36, 0.75, 0.82
Mem: 4051312k total, 1062964k used, 2988348k free, 167004k buffers
Swap: 6094840k total, 0k used, 6094840k free, 680144k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4245 root 15 0 791m 86m 10m S 39.6 2.2 1192:32 asterisk
18280 root 15 0 3812 600 516 S 2.0 0.0 0:59.00 pppoe
2582 root 15 0 5912 628 504 S 0.3 0.0 2:02.19 syslogd
18978 root 15 0 12744 1096 812 R 0.3 0.0 0:00.02 top
1 root 15 0 10352 700 588 S 0.0 0.0 0:01.14 init
2 root RT -5 0 0 0 S 0.0 0.0 0:00.01 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:31.90 ksoftirqd/0
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
5 root RT -5 0 0 0 S 0.0 0.0 0:00.01 migration/1
6 root 34 19 0 0 0 S 0.0 0.0 0:08.43 ksoftirqd/1
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
8 root RT -5 0 0 0 S 0.0 0.0 0:00.13 migration/2
9 root 34 19 0 0 0 S 0.0 0.0 2:40.56 ksoftirqd/2
10 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/2
11 root RT -5 0 0 0 S 0.0 0.0 0:00.05 migration/3
12 root 34 19 0 0 0 S 0.0 0.0 0:44.56 ksoftirqd/3
13 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/3
14 root 10 -5 0 0 0 S 0.0 0.0 0:00.02 events/0
15 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/1
16 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/2
17 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/3
18 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khelper
55 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread
62 root 10 -5 0 0 0 S 0.0 0.0 0:00.07 kblockd/0
63 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 kblockd/1
64 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/2
65 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/3
66 root 17 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid
166 root 17 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/0
167 root 18 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/1
Dave
hard to equate sip attack to ping performance.. Run mtr for a bit. Also try tcpdump or wireshark or tethereal. If you are really paranoid recycle all your passwords Sent from my iPhone On Jul 31, 2011, at 7:04 PM, "Dave George" <dgeorge at teletoneinc.com> wrote:> My asterisk server is getting bogged down every 5 minutes. My ping time is > going from 60ms to 800 ms and the call quality is bad. > > I have fail2ban running and I am using iptables. I have two ip connections > to the box. > > How can I tell if the poor performance is due to sip attacks? I don't see > any reg attempts in my asterisk cli. I use to get frequent attacks but > fail2ban seems to be taking care of that. > > See how ping time gets worst in a short space of time and server performance > at the time: > > > 64 bytes from 4.2.2.1: icmp_seq=6 ttl=55 time=87.8 ms > 64 bytes from 4.2.2.1: icmp_seq=7 ttl=55 time=99.8 ms > 64 bytes from 4.2.2.1: icmp_seq=8 ttl=55 time=107 ms > 64 bytes from 4.2.2.1: icmp_seq=9 ttl=55 time=115 ms > 64 bytes from 4.2.2.1: icmp_seq=10 ttl=55 time=120 ms > 64 bytes from 4.2.2.1: icmp_seq=11 ttl=55 time=122 ms > 64 bytes from 4.2.2.1: icmp_seq=12 ttl=55 time=123 ms > 64 bytes from 4.2.2.1: icmp_seq=13 ttl=55 time=126 ms > 64 bytes from 4.2.2.1: icmp_seq=14 ttl=55 time=122 ms > 64 bytes from 4.2.2.1: icmp_seq=15 ttl=55 time=142 ms > 64 bytes from 4.2.2.1: icmp_seq=16 ttl=55 time=142 ms > 64 bytes from 4.2.2.1: icmp_seq=17 ttl=55 time=137 ms > 64 bytes from 4.2.2.1: icmp_seq=18 ttl=55 time=186 ms > 64 bytes from 4.2.2.1: icmp_seq=19 ttl=55 time=255 ms > 64 bytes from 4.2.2.1: icmp_seq=20 ttl=55 time=310 ms > 64 bytes from 4.2.2.1: icmp_seq=21 ttl=55 time=387 ms > 64 bytes from 4.2.2.1: icmp_seq=22 ttl=55 time=445 ms > 64 bytes from 4.2.2.1: icmp_seq=23 ttl=55 time=514 ms > 64 bytes from 4.2.2.1: icmp_seq=24 ttl=55 time=583 ms > 64 bytes from 4.2.2.1: icmp_seq=25 ttl=55 time=650 ms > 64 bytes from 4.2.2.1: icmp_seq=26 ttl=55 time=715 ms > 64 bytes from 4.2.2.1: icmp_seq=27 ttl=55 time=783 ms > 64 bytes from 4.2.2.1: icmp_seq=28 ttl=55 time=821 ms > 64 bytes from 4.2.2.1: icmp_seq=29 ttl=55 time=810 ms > 64 bytes from 4.2.2.1: icmp_seq=30 ttl=55 time=832 ms > 64 bytes from 4.2.2.1: icmp_seq=31 ttl=55 time=812 ms > 64 bytes from 4.2.2.1: icmp_seq=32 ttl=55 time=821 ms > 64 bytes from 4.2.2.1: icmp_seq=33 ttl=55 time=826 ms > 64 bytes from 4.2.2.1: icmp_seq=34 ttl=55 time=815 ms > 64 bytes from 4.2.2.1: icmp_seq=35 ttl=55 time=821 ms > 64 bytes from 4.2.2.1: icmp_seq=36 ttl=55 time=824 ms > > top - 19:02:38 up 4 days, 11:26, 4 users, load average: 0.36, 0.75, 0.82 > Mem: 4051312k total, 1062964k used, 2988348k free, 167004k buffers > Swap: 6094840k total, 0k used, 6094840k free, 680144k cached > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 4245 root 15 0 791m 86m 10m S 39.6 2.2 1192:32 asterisk > 18280 root 15 0 3812 600 516 S 2.0 0.0 0:59.00 pppoe > 2582 root 15 0 5912 628 504 S 0.3 0.0 2:02.19 syslogd > 18978 root 15 0 12744 1096 812 R 0.3 0.0 0:00.02 top > 1 root 15 0 10352 700 588 S 0.0 0.0 0:01.14 init > 2 root RT -5 0 0 0 S 0.0 0.0 0:00.01 migration/0 > 3 root 34 19 0 0 0 S 0.0 0.0 0:31.90 ksoftirqd/0 > 4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0 > 5 root RT -5 0 0 0 S 0.0 0.0 0:00.01 migration/1 > 6 root 34 19 0 0 0 S 0.0 0.0 0:08.43 ksoftirqd/1 > 7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1 > 8 root RT -5 0 0 0 S 0.0 0.0 0:00.13 migration/2 > 9 root 34 19 0 0 0 S 0.0 0.0 2:40.56 ksoftirqd/2 > 10 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/2 > 11 root RT -5 0 0 0 S 0.0 0.0 0:00.05 migration/3 > 12 root 34 19 0 0 0 S 0.0 0.0 0:44.56 ksoftirqd/3 > 13 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/3 > 14 root 10 -5 0 0 0 S 0.0 0.0 0:00.02 events/0 > 15 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/1 > 16 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/2 > 17 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/3 > 18 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khelper > 55 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread > 62 root 10 -5 0 0 0 S 0.0 0.0 0:00.07 kblockd/0 > 63 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 kblockd/1 > 64 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/2 > 65 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/3 > 66 root 17 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid > 166 root 17 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/0 > 167 root 18 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/1 > > > > Dave > > > > -- > _____________________________________________________________________ > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > New to Asterisk? Join us for a live introductory webinar every Thurs: > http://www.asterisk.org/hello > > asterisk-users mailing list > To UNSUBSCRIBE or update options visit: > http://lists.digium.com/mailman/listinfo/asterisk-users
How long ago was the last block from fail2ban? What could be is that the attacker hasn't yet realized that he has been blocked and is still trying, which although blocked by iptables it is still coming down the line for attempted connections. On Sun, Jul 31, 2011 at 7:04 PM, Dave George <dgeorge at teletoneinc.com> wrote:> My asterisk server is getting bogged down every 5 minutes. ?My ping time is > going from 60ms to 800 ms and the call quality is bad. > > I have fail2ban running and I am using iptables. ?I have two ip connections > to the box. > > How can I tell if the poor performance is due to sip attacks? ? I don't see > any reg attempts in my asterisk cli. ?I use to get frequent attacks but > fail2ban seems to be taking care of that. > > See how ping time gets worst in a short space of time and server performance > at the time: > > > 64 bytes from 4.2.2.1: icmp_seq=6 ttl=55 time=87.8 ms > 64 bytes from 4.2.2.1: icmp_seq=7 ttl=55 time=99.8 ms > 64 bytes from 4.2.2.1: icmp_seq=8 ttl=55 time=107 ms > 64 bytes from 4.2.2.1: icmp_seq=9 ttl=55 time=115 ms > 64 bytes from 4.2.2.1: icmp_seq=10 ttl=55 time=120 ms > 64 bytes from 4.2.2.1: icmp_seq=11 ttl=55 time=122 ms > 64 bytes from 4.2.2.1: icmp_seq=12 ttl=55 time=123 ms > 64 bytes from 4.2.2.1: icmp_seq=13 ttl=55 time=126 ms > 64 bytes from 4.2.2.1: icmp_seq=14 ttl=55 time=122 ms > 64 bytes from 4.2.2.1: icmp_seq=15 ttl=55 time=142 ms > 64 bytes from 4.2.2.1: icmp_seq=16 ttl=55 time=142 ms > 64 bytes from 4.2.2.1: icmp_seq=17 ttl=55 time=137 ms > 64 bytes from 4.2.2.1: icmp_seq=18 ttl=55 time=186 ms > 64 bytes from 4.2.2.1: icmp_seq=19 ttl=55 time=255 ms > 64 bytes from 4.2.2.1: icmp_seq=20 ttl=55 time=310 ms > 64 bytes from 4.2.2.1: icmp_seq=21 ttl=55 time=387 ms > 64 bytes from 4.2.2.1: icmp_seq=22 ttl=55 time=445 ms > 64 bytes from 4.2.2.1: icmp_seq=23 ttl=55 time=514 ms > 64 bytes from 4.2.2.1: icmp_seq=24 ttl=55 time=583 ms > 64 bytes from 4.2.2.1: icmp_seq=25 ttl=55 time=650 ms > 64 bytes from 4.2.2.1: icmp_seq=26 ttl=55 time=715 ms > 64 bytes from 4.2.2.1: icmp_seq=27 ttl=55 time=783 ms > 64 bytes from 4.2.2.1: icmp_seq=28 ttl=55 time=821 ms > 64 bytes from 4.2.2.1: icmp_seq=29 ttl=55 time=810 ms > 64 bytes from 4.2.2.1: icmp_seq=30 ttl=55 time=832 ms > 64 bytes from 4.2.2.1: icmp_seq=31 ttl=55 time=812 ms > 64 bytes from 4.2.2.1: icmp_seq=32 ttl=55 time=821 ms > 64 bytes from 4.2.2.1: icmp_seq=33 ttl=55 time=826 ms > 64 bytes from 4.2.2.1: icmp_seq=34 ttl=55 time=815 ms > 64 bytes from 4.2.2.1: icmp_seq=35 ttl=55 time=821 ms > 64 bytes from 4.2.2.1: icmp_seq=36 ttl=55 time=824 ms > > top - 19:02:38 up 4 days, 11:26, ?4 users, ?load average: 0.36, 0.75, 0.82 > Mem: ? 4051312k total, ?1062964k used, ?2988348k free, ? 167004k buffers > Swap: ?6094840k total, ? ? ? ?0k used, ?6094840k free, ? 680144k cached > > ?PID USER ? ? ?PR ?NI ?VIRT ?RES ?SHR S %CPU %MEM ? ?TIME+ ?COMMAND > ?4245 root ? ? ?15 ? 0 ?791m ?86m ?10m S 39.6 ?2.2 ? 1192:32 asterisk > 18280 root ? ? ?15 ? 0 ?3812 ?600 ?516 S ?2.0 ?0.0 ? 0:59.00 pppoe > ?2582 root ? ? ?15 ? 0 ?5912 ?628 ?504 S ?0.3 ?0.0 ? 2:02.19 syslogd > 18978 root ? ? ?15 ? 0 12744 1096 ?812 R ?0.3 ?0.0 ? 0:00.02 top > ? ?1 root ? ? ?15 ? 0 10352 ?700 ?588 S ?0.0 ?0.0 ? 0:01.14 init > ? ?2 root ? ? ?RT ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.01 migration/0 > ? ?3 root ? ? ?34 ?19 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:31.90 ksoftirqd/0 > ? ?4 root ? ? ?RT ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 watchdog/0 > ? ?5 root ? ? ?RT ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.01 migration/1 > ? ?6 root ? ? ?34 ?19 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:08.43 ksoftirqd/1 > ? ?7 root ? ? ?RT ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 watchdog/1 > ? ?8 root ? ? ?RT ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.13 migration/2 > ? ?9 root ? ? ?34 ?19 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 2:40.56 ksoftirqd/2 > ? 10 root ? ? ?RT ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 watchdog/2 > ? 11 root ? ? ?RT ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.05 migration/3 > ? 12 root ? ? ?34 ?19 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:44.56 ksoftirqd/3 > ? 13 root ? ? ?RT ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 watchdog/3 > ? 14 root ? ? ?10 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.02 events/0 > ? 15 root ? ? ?10 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 events/1 > ? 16 root ? ? ?10 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 events/2 > ? 17 root ? ? ?10 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 events/3 > ? 18 root ? ? ?10 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 khelper > ? 55 root ? ? ?10 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 kthread > ? 62 root ? ? ?10 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.07 kblockd/0 > ? 63 root ? ? ?10 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.01 kblockd/1 > ? 64 root ? ? ?10 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 kblockd/2 > ? 65 root ? ? ?10 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 kblockd/3 > ? 66 root ? ? ?17 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 kacpid > ?166 root ? ? ?17 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 cqueue/0 > ?167 root ? ? ?18 ?-5 ? ? 0 ? ?0 ? ?0 S ?0.0 ?0.0 ? 0:00.00 cqueue/1 > > > > Dave > > > > -- > _____________________________________________________________________ > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > New to Asterisk? Join us for a live introductory webinar every Thurs: > ? ? ? ? ? ? ? http://www.asterisk.org/hello > > asterisk-users mailing list > To UNSUBSCRIBE or update options visit: > ? http://lists.digium.com/mailman/listinfo/asterisk-users >