Dovid Bender writes:> The tcpdump that you are running is on the Asterisk box or via port > mirroring?It's on the asterisk box itself. I've already replaced the network card - no change. Thanks, Roel> Regards, > > Dovid > > -----Original Message----- > From: Roel van Meer <roel at 1afa.com> > Sender: asterisk-users-bounces at lists.digium.comDate: Thu, 31 Mar 2016 > 13:34:51 > To: <asterisk-users at lists.digium.com> > Reply-To: Asterisk Users Mailing List - Non-Commercial Discussion > <asterisk-users at lists.digium.com> > Subject: [asterisk-users] Lost outgoing SIP packets > > Hi list! > > I have a problem where SIP packets sent by Asterisk do not hit the wire, and > I don't know what could cause this. > > I'm running Asterisk 1.8.28_cert5 with full SIP debug. At the same time, I'm > doing a tcpdump of the traffic on the network interface. I can see in the SIP > debug log that asterisk is sending packets. Most of the time, I can see > those packets in the tcpdump, as you would expect. > However, sometimes Asterisk sends a packet that *does not show up* in the > tcpdump. Asterisk then does several retransmits (that also don't show up). > The next packet that is not a retransmit does show up again. > > This causes Asterisk to log the peer it was sending packets to temporarily > as Lagged or unreachable. > > There is no outgoing firewall on this box. > > Could anyone give me some pointers where to look? > > If Asterisk logs "VERBOSE[13019] chan_sip.c: Reliably Transmitting (NAT) to > x.x.x.x:" you would expect to see that packet in a tcpdump trace, right? > What could cause this not to be so? Are there network statistics I could > look at? Is there a counter in /proc or /sys for problems with sending > packets? Anything? > > If more information is necessary please do let me know. > > Thanks a lot in advance, > > Roel > > -- > _____________________________________________________________________ > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > New to Asterisk? Join us for a live introductory webinar every Thurs: > http://www.asterisk.org/hello > > asterisk-users mailing list > To UNSUBSCRIBE or update options visit: > http://lists.digium.com/mailman/listinfo/asterisk-users > -- > _____________________________________________________________________ > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > New to Asterisk? Join us for a live introductory webinar every Thurs: > http://www.asterisk.org/hello > > asterisk-users mailing list > To UNSUBSCRIBE or update options visit: > http://lists.digium.com/mailman/listinfo/asterisk-users
Just guessing I would verify that the out of : iptables -L -nv Shows no dropped packets, try disabling selinux as well as look at the limits of the asterisk pid (cat /proc/<Asterisk PID>/limits). I know the defualt for rhel is 1024 which was never enough for us. Regards, Dovid -----Original Message----- From: Roel van Meer <roel at 1afa.com> Date: Thu, 31 Mar 2016 14:10:48 To: <dovid at telecurve.com>; Asterisk Users Mailing List - Non-Commercial Discussion<asterisk-users at lists.digium.com> Subject: Re: [asterisk-users] Lost outgoing SIP packets Dovid Bender writes:> The tcpdump that you are running is on the Asterisk box or via port > mirroring?It's on the asterisk box itself. I've already replaced the network card - no change. Thanks, Roel> Regards, > > Dovid > > -----Original Message----- > From: Roel van Meer <roel at 1afa.com> > Sender: asterisk-users-bounces at lists.digium.comDate: Thu, 31 Mar 2016 > 13:34:51 > To: <asterisk-users at lists.digium.com> > Reply-To: Asterisk Users Mailing List - Non-Commercial Discussion > <asterisk-users at lists.digium.com> > Subject: [asterisk-users] Lost outgoing SIP packets > > Hi list! > > I have a problem where SIP packets sent by Asterisk do not hit the wire, and > I don't know what could cause this. > > I'm running Asterisk 1.8.28_cert5 with full SIP debug. At the same time, I'm > doing a tcpdump of the traffic on the network interface. I can see in the SIP > debug log that asterisk is sending packets. Most of the time, I can see > those packets in the tcpdump, as you would expect. > However, sometimes Asterisk sends a packet that *does not show up* in the > tcpdump. Asterisk then does several retransmits (that also don't show up). > The next packet that is not a retransmit does show up again. > > This causes Asterisk to log the peer it was sending packets to temporarily > as Lagged or unreachable. > > There is no outgoing firewall on this box. > > Could anyone give me some pointers where to look? > > If Asterisk logs "VERBOSE[13019] chan_sip.c: Reliably Transmitting (NAT) to > x.x.x.x:" you would expect to see that packet in a tcpdump trace, right? > What could cause this not to be so? Are there network statistics I could > look at? Is there a counter in /proc or /sys for problems with sending > packets? Anything? > > If more information is necessary please do let me know. > > Thanks a lot in advance, > > Roel > > -- > _____________________________________________________________________ > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > New to Asterisk? Join us for a live introductory webinar every Thurs: > http://www.asterisk.org/hello > > asterisk-users mailing list > To UNSUBSCRIBE or update options visit: > http://lists.digium.com/mailman/listinfo/asterisk-users > -- > _____________________________________________________________________ > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > New to Asterisk? Join us for a live introductory webinar every Thurs: > http://www.asterisk.org/hello > > asterisk-users mailing list > To UNSUBSCRIBE or update options visit: > http://lists.digium.com/mailman/listinfo/asterisk-users
Dovid Bender writes:> Just guessing I would verify that the out of : iptables -L -nv > Shows no dropped packets, try disabling selinux as well as look at the > limits of the asterisk pid (cat /proc/<Asterisk PID>/limits). I know the > defualt for rhel is 1024 which was never enough for us.Thanks for the hints. Selinux is disabled, there is no outgoing firewall (anymore) on this box, and the limits seems fine: 200637 open files. Ifconfig output looks like this: root at communiceer:~# ifconfig eth1 eth1 Link encap:Ethernet HWaddr b4:99:ba:a9:3e:e5 inet addr:x.x.x.x Bcast:x.x.x.127 Mask:255.255.255.128 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5967421 errors:0 dropped:21425 overruns:0 frame:0 TX packets:6085933 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1223605260 (1.1 GiB) TX bytes:2096293903 (1.9 GiB) Interrupt:17 Memory:fbfe0000-fc000000 I was thinking maybe there's a problem with the transmit queue, but 1000 is the default value for txqueuelen and I have never needed to change it. I have the default queueing discipline: root at communiceer:~# tc qdisc show dev eth1 qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1 The output of ethtool also looks good: root at communiceer:~# ethtool eth1 Settings for eth1: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised pause frame use: No Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 1 Transceiver: internal Auto-negotiation: on MDI-X: on Supports Wake-on: pumbg Wake-on: g Current message level: 0x00000007 (7) drv probe link Link detected: yes And the nic stats also look good: root at communiceer:~# ethtool -S eth1 NIC statistics: rx_packets: 6071960 tx_packets: 6189424 rx_bytes: 1244435132 tx_bytes: 2117335817 rx_broadcast: 293751 tx_broadcast: 193 rx_multicast: 29827 tx_multicast: 0 rx_errors: 0 tx_errors: 0 tx_dropped: 0 multicast: 29827 collisions: 0 rx_length_errors: 0 rx_over_errors: 0 rx_crc_errors: 0 rx_frame_errors: 0 rx_no_buffer_count: 0 rx_missed_errors: 0 tx_aborted_errors: 0 tx_carrier_errors: 0 tx_fifo_errors: 0 tx_heartbeat_errors: 0 tx_window_errors: 0 tx_abort_late_coll: 0 tx_deferred_ok: 0 tx_single_coll_ok: 0 tx_multi_coll_ok: 0 tx_timeout_count: 0 tx_restart_queue: 0 rx_long_length_errors: 0 rx_short_length_errors: 0 rx_align_errors: 0 tx_tcp_seg_good: 37559 tx_tcp_seg_failed: 0 rx_flow_control_xon: 0 rx_flow_control_xoff: 0 tx_flow_control_xon: 0 tx_flow_control_xoff: 0 rx_csum_offload_good: 3447739 rx_csum_offload_errors: 2 rx_header_split: 0 alloc_rx_buff_failed: 0 tx_smbus: 0 rx_smbus: 0 dropped_smbus: 0 rx_dma_failed: 0 tx_dma_failed: 0 rx_hwtstamp_cleared: 0 uncorr_ecc_errors: 0 corr_ecc_errors: 0 tx_hwtstamp_timeouts: 0 So I really don't know where to look elsewhere.. Thanks, Roel> > -----Original Message----- > From: Roel van Meer <roel at 1afa.com> > Date: Thu, 31 Mar 2016 14:10:48 > To: <dovid at telecurve.com>; Asterisk Users Mailing List - Non-Commercial > Discussion<asterisk-users at lists.digium.com> > Subject: Re: [asterisk-users] Lost outgoing SIP packets > > Dovid Bender writes: > > > The tcpdump that you are running is on the Asterisk box or via port > > mirroring? > > It's on the asterisk box itself. > > I've already replaced the network card - no change. > > Thanks, > > Roel > > > > Regards, > > > > Dovid > > > > -----Original Message----- > > From: Roel van Meer <roel at 1afa.com> > > Sender: asterisk-users-bounces at lists.digium.comDate: Thu, 31 Mar 2016 > > 13:34:51 > > To: <asterisk-users at lists.digium.com> > > Reply-To: Asterisk Users Mailing List - Non-Commercial Discussion > > <asterisk-users at lists.digium.com> > > Subject: [asterisk-users] Lost outgoing SIP packets > > > > Hi list! > > > > I have a problem where SIP packets sent by Asterisk do not hit the wire, > and > > I don't know what could cause this. > > > > I'm running Asterisk 1.8.28_cert5 with full SIP debug. At the same time, > I'm > > doing a tcpdump of the traffic on the network interface. I can see in the > SIP > > debug log that asterisk is sending packets. Most of the time, I can see > > those packets in the tcpdump, as you would expect. > > However, sometimes Asterisk sends a packet that *does not show up* in the > > tcpdump. Asterisk then does several retransmits (that also don't show up). > > The next packet that is not a retransmit does show up again. > > > > This causes Asterisk to log the peer it was sending packets to temporarily > > as Lagged or unreachable. > > > > There is no outgoing firewall on this box. > > > > Could anyone give me some pointers where to look? > > > > If Asterisk logs "VERBOSE[13019] chan_sip.c: Reliably Transmitting (NAT) to > > x.x.x.x:" you would expect to see that packet in a tcpdump trace, right? > > What could cause this not to be so? Are there network statistics I could > > look at? Is there a counter in /proc or /sys for problems with sending > > packets? Anything? > > > > If more information is necessary please do let me know. > > > > Thanks a lot in advance, > > > > Roel > > > > -- > > _____________________________________________________________________ > > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > > New to Asterisk? Join us for a live introductory webinar every Thurs: > > http://www.asterisk.org/hello > > > > asterisk-users mailing list > > To UNSUBSCRIBE or update options visit: > > http://lists.digium.com/mailman/listinfo/asterisk-users > > -- > > _____________________________________________________________________ > > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > > New to Asterisk? Join us for a live introductory webinar every Thurs: > > http://www.asterisk.org/hello > > > > asterisk-users mailing list > > To UNSUBSCRIBE or update options visit: > > http://lists.digium.com/mailman/listinfo/asterisk-users
Hi Roel Just guessing: do you have conntrack enabled? If not, "modprobe nf_conntrack_netlink" (you can remove it and its dependencies later) What are the outputs of sysctl net.netfilter.nf_conntrack_count and sysctl net.netfilter.nf_conntrack_max when the problem shows up? cheers Ethy On Thu, 31 Mar 2016 12:17:12 +0000 "Dovid Bender" <dovid at telecurve.com> wrote:> Just guessing I would verify that the out of : iptables -L -nv > Shows no dropped packets, try disabling selinux as well as look at the limits > of the asterisk pid (cat /proc/<Asterisk PID>/limits). I know the defualt for > rhel is 1024 which was never enough for us. > > > Regards, > > Dovid > > -----Original Message----- > From: Roel van Meer <roel at 1afa.com> > Date: Thu, 31 Mar 2016 14:10:48 > To: <dovid at telecurve.com>; Asterisk Users Mailing List - Non-Commercial > Discussion<asterisk-users at lists.digium.com> Subject: Re: [asterisk-users] > Lost outgoing SIP packets > > Dovid Bender writes: > > > The tcpdump that you are running is on the Asterisk box or via port > > mirroring? > > It's on the asterisk box itself. > > I've already replaced the network card - no change. > > Thanks, > > Roel > > > > Regards, > > > > Dovid > > > > -----Original Message----- > > From: Roel van Meer <roel at 1afa.com> > > Sender: asterisk-users-bounces at lists.digium.comDate: Thu, 31 Mar 2016 > > 13:34:51 > > To: <asterisk-users at lists.digium.com> > > Reply-To: Asterisk Users Mailing List - Non-Commercial Discussion > > <asterisk-users at lists.digium.com> > > Subject: [asterisk-users] Lost outgoing SIP packets > > > > Hi list! > > > > I have a problem where SIP packets sent by Asterisk do not hit the wire, and > > I don't know what could cause this. > > > > I'm running Asterisk 1.8.28_cert5 with full SIP debug. At the same time, I'm > > doing a tcpdump of the traffic on the network interface. I can see in the > > SIP debug log that asterisk is sending packets. Most of the time, I can see > > those packets in the tcpdump, as you would expect. > > However, sometimes Asterisk sends a packet that *does not show up* in the > > tcpdump. Asterisk then does several retransmits (that also don't show up). > > The next packet that is not a retransmit does show up again. > > > > This causes Asterisk to log the peer it was sending packets to temporarily > > as Lagged or unreachable. > > > > There is no outgoing firewall on this box. > > > > Could anyone give me some pointers where to look? > > > > If Asterisk logs "VERBOSE[13019] chan_sip.c: Reliably Transmitting (NAT) to > > x.x.x.x:" you would expect to see that packet in a tcpdump trace, right? > > What could cause this not to be so? Are there network statistics I could > > look at? Is there a counter in /proc or /sys for problems with sending > > packets? Anything? > > > > If more information is necessary please do let me know. > > > > Thanks a lot in advance, > > > > Roel > > > > -- > > _____________________________________________________________________ > > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > > New to Asterisk? Join us for a live introductory webinar every Thurs: > > http://www.asterisk.org/hello > > > > asterisk-users mailing list > > To UNSUBSCRIBE or update options visit: > > http://lists.digium.com/mailman/listinfo/asterisk-users > > -- > > _____________________________________________________________________ > > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > > New to Asterisk? Join us for a live introductory webinar every Thurs: > > http://www.asterisk.org/hello > > > > asterisk-users mailing list > > To UNSUBSCRIBE or update options visit: > > http://lists.digium.com/mailman/listinfo/asterisk-users > -- > _____________________________________________________________________ > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > New to Asterisk? Join us for a live introductory webinar every Thurs: > http://www.asterisk.org/hello > > asterisk-users mailing list > To UNSUBSCRIBE or update options visit: > http://lists.digium.com/mailman/listinfo/asterisk-users
On Thu, 31 Mar 2016, Dovid Bender wrote:> Just guessing I would verify that the out of : iptables -L -nv Shows no > dropped packets...Doesn't tcpdump 'see' packets before iptables? -- Thanks in advance, ------------------------------------------------------------------------- Steve Edwards sedwards at sedwards.com Voice: +1-760-468-3867 PST https://www.linkedin.com/in/steve-edwards-4244281