tinc - May 2016 - Lots of Flushing x bytes to y would block messages

We run tinc in a linux environment in which it sits there waiting for
connections from the clients. All clients are configured to only have one
ConnectTo which points to this server.

We're seeing in the server log that as soon as a client's connection is
activated, a whole bunch of "Flushing x bytes to that host would
block" is
logged and the whole vpn is bogged down and has become non-responsive.
>From a Jun 27, 2013 "Metadata socket read error" thread, Gus
suggested that
it may be caused by the router in front of the server that ran out of
memory keeping track of the outgoing connections. The server is indeed
behind a NAT router. It's a reasonably (ASUS) high end consumer grade
router.

We have tried adding PingInterval in the host configuration file, but it
doesn't seem to resolve the problem. Is there anything else that we can do
to stop the vpn from crashing. As it is, when any one of the client is
logged to be flushing, the whole vpn becomes unusable.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20160504/3a6d9274/attachment.html>

On Wed, May 04, 2016 at 07:10:47AM +0800, Terry T wrote:
> We run tinc in a linux environment in which it sits there waiting for
> connections from the clients. All clients are configured to only have one
> ConnectTo which points to this server.
> 
> We're seeing in the server log that as soon as a client's
connection is
> activated, a whole bunch of "Flushing x bytes to that host would
block" is
> logged and the whole vpn is bogged down and has become non-responsive.
This means that the bandwidth to that client is so low that the TCP
buffer on the server is full. It should not cause communication between
the other clients to slow down though...
> From a Jun 27, 2013 "Metadata socket read error" thread, Gus
suggested that
> it may be caused by the router in front of the server that ran out of
> memory keeping track of the outgoing connections. The server is indeed
> behind a NAT router. It's a reasonably (ASUS) high end consumer grade
> router.
This is not the same problem.
> We have tried adding PingInterval in the host configuration file, but it
> doesn't seem to resolve the problem. Is there anything else that we can
do
> to stop the vpn from crashing. As it is, when any one of the client is
> logged to be flushing, the whole vpn becomes unusable.
Hm, does non-VPN traffic to/from the server also get slow when this
happens?

-- 
Met vriendelijke groet / with kind regards,
     Guus Sliepen <guus at tinc-vpn.org>

The server has a 1G symmetrical fibre line. It has been speedtested to
various local servers to be close to 800-900M. When there is only a single
client, there isn't much problem and as soon as the connection is made, the
ping time through to tunnel is a respectable 30ms. As soon as a few more
clients are connected, ping time degrades to hundreds and sometimes seconds
and with dropped packets. Dataflow would appear stopped and the tinc log on
the server would literally filled with hundreds of flush....would block
messages. All hosts are running latest tinc-1.0 stable.

The server is configured as a bridge and is relaying multicasts
continuously.  Below is the server configuration.

Name = tserver
AddressFamily = ipv4
BindToAddress = 192.168.21.254 30000
KeyExpire = 28800
ReplayWindow = 0
DeviceStandby = no
DeviceType = tap
DirectOnly = yes
Mode = hub
ProcessPriority = high
ClampMSS = yes
Cipher = none
Digest = none
MACLength = 0
PMTUDiscovery = yes

I have taken out what I believe is performance sapping options in an effort
to boost performance.

All clients (Windows 7) configuration is identical save its own name.

Name = <client name>
ConnectTo = tserver
AddressFamily = ipv4
KeyExpire = 28800
ReplayWindow = 0
Broadcast = direct
DirectOnly = yes
Mode = hub
ClampMSS = yes
Cipher = none
Digest = none
MACLength = 0
PMTUDiscovery = yes
Interface = win-Tap



On Fri, May 6, 2016 at 3:25 PM, Guus Sliepen <guus at tinc-vpn.org> wrote:
> On Wed, May 04, 2016 at 07:10:47AM +0800, Terry T wrote:
>
> > We run tinc in a linux environment in which it sits there waiting for
> > connections from the clients. All clients are configured to only have
one
> > ConnectTo which points to this server.
> >
> > We're seeing in the server log that as soon as a client's
connection is
> > activated, a whole bunch of "Flushing x bytes to that host would
block"
> is
> > logged and the whole vpn is bogged down and has become non-responsive.
>
> This means that the bandwidth to that client is so low that the TCP
> buffer on the server is full. It should not cause communication between
> the other clients to slow down though...
>
> > From a Jun 27, 2013 "Metadata socket read error" thread, Gus
suggested
> that
> > it may be caused by the router in front of the server that ran out of
> > memory keeping track of the outgoing connections. The server is indeed
> > behind a NAT router. It's a reasonably (ASUS) high end consumer
grade
> > router.
>
> This is not the same problem.
>
> > We have tried adding PingInterval in the host configuration file, but
it
> > doesn't seem to resolve the problem. Is there anything else that
we can
> do
> > to stop the vpn from crashing. As it is, when any one of the client is
> > logged to be flushing, the whole vpn becomes unusable.
>
> Hm, does non-VPN traffic to/from the server also get slow when this
> happens?
>
> --
> Met vriendelijke groet / with kind regards,
>      Guus Sliepen <guus at tinc-vpn.org>
> _______________________________________________
> tinc mailing list
> tinc at tinc-vpn.org
> https://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20160506/c4a1b32d/attachment.html>